2022-05-30T19:31:37.621 INFO:root:teuthology version: 1.1.1.dev341+g1b30281f 2022-05-30T19:31:37.795 INFO:teuthology.lock.ops:Start node 'smithi035.front.sepia.ceph.com' reimaging 2022-05-30T19:31:37.795 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: reset os type and version on server 2022-05-30T19:31:37.796 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2022-05-30T19:31:37.817 INFO:teuthology.lock.ops:Node 'smithi035.front.sepia.ceph.com' reimaging is complete 2022-05-30T19:31:37.817 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging 2022-05-30T19:31:37.817 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server 2022-05-30T19:31:37.817 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2022-05-30T19:31:37.840 INFO:teuthology.lock.ops:Node 'smithi157.front.sepia.ceph.com' reimaging is complete 2022-05-30T19:31:37.841 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2022-05-30T19:31:37.841 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2022-05-30T19:31:37.841 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2022-05-30T19:31:37.862 INFO:teuthology.lock.ops:Node 'smithi191.front.sepia.ceph.com' reimaging is complete 2022-05-30T19:31:38.040 INFO:teuthology.provision.fog.smithi035:Scheduling deploy of centos 8.stream 2022-05-30T19:31:38.059 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of centos 8.stream 2022-05-30T19:31:38.078 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 8.stream 2022-05-30T19:31:38.434 INFO:teuthology.orchestra.console:Power off smithi035 2022-05-30T19:31:38.449 INFO:teuthology.orchestra.console:Power off smithi157 2022-05-30T19:31:38.464 INFO:teuthology.orchestra.console:Power off smithi191 2022-05-30T19:31:46.832 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2022-05-30T19:31:46.933 INFO:teuthology.orchestra.console:Power on smithi191 2022-05-30T19:31:50.932 INFO:teuthology.orchestra.console:Power off for smithi035 completed 2022-05-30T19:31:50.937 INFO:teuthology.orchestra.console:Power off for smithi157 completed 2022-05-30T19:31:51.033 INFO:teuthology.orchestra.console:Power on smithi035 2022-05-30T19:31:51.041 INFO:teuthology.orchestra.console:Power on smithi157 2022-05-30T19:31:55.294 INFO:teuthology.orchestra.console:Power on for smithi035 completed 2022-05-30T19:31:55.301 INFO:teuthology.orchestra.console:Power on for smithi191 completed 2022-05-30T19:31:55.308 INFO:teuthology.orchestra.console:Power on for smithi157 completed 2022-05-30T19:31:55.396 INFO:teuthology.provision.fog.smithi035:Waiting for deploy to finish 2022-05-30T19:31:55.402 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2022-05-30T19:31:55.410 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish 2022-05-30T19:36:09.310 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-30T19:36:09.593 INFO:teuthology.provision.fog.smithi035:Node is ready 2022-05-30T19:36:09.614 INFO:teuthology.orchestra.run.smithi035.stdout:smithi035.front.sepia.ceph.com 2022-05-30T19:36:09.675 INFO:teuthology.orchestra.run.smithi035.stdout:172.21.15.35 smithi035.front.sepia.ceph.com smithi035 2022-05-30T19:36:10.062 INFO:teuthology.provision.fog.smithi035:Deploy complete! 2022-05-30T19:36:33.445 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-30T19:36:33.742 INFO:teuthology.provision.fog.smithi191:Node is ready 2022-05-30T19:36:33.760 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2022-05-30T19:36:33.818 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2022-05-30T19:36:34.172 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2022-05-30T19:37:21.804 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-30T19:37:22.068 INFO:teuthology.provision.fog.smithi157:Node is ready 2022-05-30T19:37:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com 2022-05-30T19:37:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157 2022-05-30T19:37:22.496 INFO:teuthology.provision.fog.smithi157:Deploy complete! 2022-05-30T19:37:22.705 INFO:teuthology.lock.ops:Checking smithi035.front.sepia.ceph.com 2022-05-30T19:37:22.706 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-30T19:37:22.730 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com 2022-05-30T19:37:22.730 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-30T19:37:22.748 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com 2022-05-30T19:37:22.748 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-30T19:37:22.769 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: set os type and version on server 2022-05-30T19:37:22.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-05-30T19:37:23.047 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2022-05-30T19:37:23.106 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Stream" 2022-05-30T19:37:23.106 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8" 2022-05-30T19:37:23.106 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31" 2022-05-30T19:37:23.107 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-30T19:37:23.108 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://centos.org/" 2022-05-30T19:37:23.108 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-30T19:37:23.108 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-30T19:37:23.108 INFO:teuthology.orchestra.run.smithi035.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-30T19:37:23.110 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2022-05-30T19:37:23.131 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server 2022-05-30T19:37:23.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2022-05-30T19:37:23.284 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2022-05-30T19:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2022-05-30T19:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8" 2022-05-30T19:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2022-05-30T19:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2022-05-30T19:37:23.341 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-30T19:37:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-30T19:37:23.343 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2022-05-30T19:37:23.364 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server 2022-05-30T19:37:23.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2022-05-30T19:37:23.640 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2022-05-30T19:37:23.699 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2022-05-30T19:37:23.699 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2022-05-30T19:37:23.699 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2022-05-30T19:37:23.700 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-30T19:37:23.701 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2022-05-30T19:37:23.702 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-30T19:37:23.702 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-30T19:37:23.702 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-30T19:37:23.703 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2022-05-30T19:37:23.887 INFO:teuthology.dispatcher.supervisor:Running job 6853673 2022-05-30T19:37:23.887 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2022-05-30_19:19:24-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6853673 --name rishabh-2022-05-30_19:19:24-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi --description fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} -- /home/teuthworker/archive/rishabh-2022-05-30_19:19:24-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6853673/orig.config.yaml 2022-05-30T19:37:23.893 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2022-05-30_19:19:24-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6853673 2022-05-30T19:37:23.893 INFO:teuthology.dispatcher.supervisor:Job PID: 11252 2022-05-30T19:37:23.893 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-05-31T02:07:52.103 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-05-31T02:07:52.280 INFO:teuthology.kill:Killing Pids: {11252} 2022-05-31T02:07:53.626 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-05-31T02:07:53.627 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['client.0'] 2022-05-31T02:07:53.627 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['client.1'] 2022-05-31T02:07:53.627 INFO:teuthology.misc:Compressing logs... 2022-05-31T02:07:53.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-05-31T02:07:53.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2022-05-31T02:07:54.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2022-05-31T02:07:54.417 INFO:teuthology.misc:Compressing logs... 2022-05-31T02:07:54.613 INFO:teuthology.misc:Compressing logs... 2022-05-31T02:10:32.688 INFO:teuthology.kill:No teuthology processes running 2022-05-31T02:10:34.146 INFO:teuthology.kill:Nuking machines: ['smithi035', 'smithi157', 'smithi191'] 2022-05-31T02:10:35.248 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-05-31T02:10:35.248 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-05-31T02:10:35.248 INFO:teuthology.kill:2022-05-31 02:10:35,247.247 INFO:teuthology.nuke:targets: 2022-05-31T02:10:35.248 INFO:teuthology.kill: smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaLpr0L8I7J8dzzAbSwNRRN6t91Z6oNzhbTH+rJqUF/2U7yC6lDVHZtyWgSbKmhDAT+mziMhTLEe4WKwRp8vgXVNdpB4TK88ACi/lg8yXfK25bF8aqRO/aYWqjBB4uk4KAV2TijYY8S8QUnetio0YS2Z+4Yowz41SAnKYzHrkz5bxjg0266nPNTafO8LNYIT8gBwOZID93ttSrBct9w2DIOIUDhy1/d1QNTkpFjMmJWtMcexeBJNDR8qb5Exmp8cpNUlMyozNG8ewDGC1PiGqukd0XMSEyAS1yR/uKgNIRLgXHIVaSQTd6Qxwgdr9kAKwTFv1JFm4i6CagTF3/IZkeAz38YYuIlijh3lqAKLgwIr9gxscBGC3qiCBLe8g0xlsufZvRwKzozKHvj0Q+M1E9by3lmDgHiOTcNueaXs71t+288QjOmbJ3lJdm1KknrX7VT2swqJ0jkMsTyjfxSW7ZIhMccC7Q9gV1x0rXxPSFqXRacTyXXvYt8XkpHqUJ48= 2022-05-31T02:10:35.249 INFO:teuthology.kill: smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBDuOMAyGxx98emQVI4v7aIfu44tiVeNQgUgeansS+zn5FuTPCjkjZTx2BbvleXtjYcDp+G14qNuZwfLEEdhpMOT5FWzrWERLDZwPUdADs0WrwUC5FyIbqsTPWP/uYMPrZg1NRn2eXGvBS0juJZRmP9WZYC1dUNidVOJ0LDt4VX3vULfzo7YqEmIwW9EWM2P9g4dZKewoGnYL+iBvakWcBndN8RcqKYFxrYaNfiUHdIVqfPTVAyi7QDNM5IjdlHt14pG+9YqJJXCaEjtKJZkzGlV8AEpywTgCwH5yanUeBB+4sNHOmVKzRn468L9gBIUZHm/3elB4QFn7HVfVVai4JOqr3pC4hweQscbHI9AGiKzZGz1wxbZHnwZGOmsdvr5Lsm7fDTWQ8zIteGh4Y4TWyO/d2rmnTm52RblUxW6sqNy4qbXGHlLMrxHKL2AxzVCVOxbYXazhSojc8I+ozIJBHpvVqLWURwgtj6TTICGFQphW1OjWHBRtk2UT2lekH78c= 2022-05-31T02:10:35.249 INFO:teuthology.kill: smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFF5n7zsbYZksaU0He2nL5bDamazJGlZq+jpReMPH23bhKzJ9a825xHKwkn7HfCwSVXym0IpxFrO7sI+cNVkgXJCiVI6GyCDuOplm3tZmbCH3WpMemq+XmDllq0aVYamjHZOYCQlrvPgUG/rrKxLf7kHhbXdjFaPhwrc3PEY5BNfIpLuPJ4PdXM/1ZsgtkzdZ8xDmYYv9UVqPwFtmIwbmMHJ5+XDpx0sOUr8me4oN+ldttvAOtDuNLqpDBT5Zmgum63GEWiHiDIS7FR/TVol+WcYTQxMRKAo1gtlWLTUPGV0Mc3V4dd4u+dzsLqCnw+FQdTY1VkMUomfaDCVWmdRc6+sMvvzllS7DvqH9haFsVwrs673t1z/N4SSzUm2n9fVXFbCrIXeBFR/TAx0fUnhsNHxzqb90Anmqg4aTRALY5IPScgh+m0nHm/M9tiLNLS2ZnNBY0sNBeidkXEqopHqNd4JWV2RiSWpugDmAn7NEKfsIo2WsZYGs86nJwA/X9T50= 2022-05-31T02:10:35.282 INFO:teuthology.kill:2022-05-31 02:10:35,281.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-31T02:10:35.285 INFO:teuthology.kill:2022-05-31 02:10:35,284.284 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-31T02:10:35.288 INFO:teuthology.kill:2022-05-31 02:10:35,287.287 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-31T02:10:35.373 INFO:teuthology.kill:2022-05-31 02:10:35,372.372 INFO:teuthology.orchestra.console:Power off smithi035 2022-05-31T02:10:35.387 INFO:teuthology.kill:2022-05-31 02:10:35,386.386 INFO:teuthology.orchestra.console:Power off smithi157 2022-05-31T02:10:35.399 INFO:teuthology.kill:2022-05-31 02:10:35,399.399 INFO:teuthology.orchestra.console:Power off smithi191 2022-05-31T02:10:43.758 INFO:teuthology.kill:2022-05-31 02:10:43,758.758 INFO:teuthology.orchestra.console:Power off for smithi157 completed 2022-05-31T02:10:43.910 INFO:teuthology.kill:2022-05-31 02:10:43,909.909 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com 2022-05-31T02:10:47.867 INFO:teuthology.kill:2022-05-31 02:10:47,867.867 INFO:teuthology.orchestra.console:Power off for smithi035 completed 2022-05-31T02:10:47.872 INFO:teuthology.kill:2022-05-31 02:10:47,872.872 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2022-05-31T02:10:48.026 INFO:teuthology.kill:2022-05-31 02:10:48,026.026 INFO:teuthology.lock.ops:unlocked: smithi191.front.sepia.ceph.com 2022-05-31T02:10:48.029 INFO:teuthology.kill:2022-05-31 02:10:48,029.029 INFO:teuthology.lock.ops:unlocked: smithi035.front.sepia.ceph.com 2022-05-31T02:12:48.375 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15