2022-10-31T23:43:37.040 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-31T23:43:37.207 INFO:teuthology.lock.ops:Start node 'smithi043.front.sepia.ceph.com' reimaging 2022-10-31T23:43:37.209 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: reset os type and version on server 2022-10-31T23:43:37.209 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2022-10-31T23:43:37.227 INFO:teuthology.lock.ops:Node 'smithi043.front.sepia.ceph.com' reimaging is complete 2022-10-31T23:43:37.228 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging 2022-10-31T23:43:37.228 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server 2022-10-31T23:43:37.229 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-10-31T23:43:37.249 INFO:teuthology.lock.ops:Node 'smithi099.front.sepia.ceph.com' reimaging is complete 2022-10-31T23:43:37.249 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging 2022-10-31T23:43:37.250 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server 2022-10-31T23:43:37.250 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-10-31T23:43:37.269 INFO:teuthology.lock.ops:Node 'smithi102.front.sepia.ceph.com' reimaging is complete 2022-10-31T23:43:37.444 INFO:teuthology.provision.fog.smithi043:Scheduling deploy of centos 8.stream 2022-10-31T23:43:37.453 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 8.stream 2022-10-31T23:43:37.470 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream 2022-10-31T23:43:37.790 INFO:teuthology.orchestra.console:Power off smithi099 2022-10-31T23:43:37.805 INFO:teuthology.orchestra.console:Power off smithi102 2022-10-31T23:43:37.816 INFO:teuthology.orchestra.console:Power off smithi043 2022-10-31T23:43:37.946 INFO:teuthology.orchestra.console:Power off for smithi099 completed 2022-10-31T23:43:37.953 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-10-31T23:43:37.958 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2022-10-31T23:43:38.048 INFO:teuthology.orchestra.console:Power on smithi099 2022-10-31T23:43:38.056 INFO:teuthology.orchestra.console:Power on smithi102 2022-10-31T23:43:38.062 INFO:teuthology.orchestra.console:Power on smithi043 2022-10-31T23:43:42.314 INFO:teuthology.orchestra.console:Power on for smithi102 completed 2022-10-31T23:43:42.415 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish 2022-10-31T23:43:46.427 INFO:teuthology.orchestra.console:Power on for smithi043 completed 2022-10-31T23:43:46.465 INFO:teuthology.orchestra.console:Power on for smithi099 completed 2022-10-31T23:43:46.529 INFO:teuthology.provision.fog.smithi043:Waiting for deploy to finish 2022-10-31T23:43:46.566 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish 2022-10-31T23:47:50.394 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-31T23:47:51.680 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-31T23:47:51.950 INFO:teuthology.provision.fog.smithi043:Node is ready 2022-10-31T23:47:51.970 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043.front.sepia.ceph.com 2022-10-31T23:47:52.031 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43 smithi043.front.sepia.ceph.com smithi043 2022-10-31T23:47:52.383 INFO:teuthology.provision.fog.smithi043:Deploy complete! 2022-10-31T23:47:54.197 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-10-31T23:48:15.733 INFO:teuthology.provision.fog.smithi099:Node is ready 2022-10-31T23:48:15.757 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com 2022-10-31T23:48:15.819 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099 2022-10-31T23:48:16.184 INFO:teuthology.provision.fog.smithi099:Deploy complete! 2022-10-31T23:48:19.051 INFO:teuthology.provision.fog.smithi102:Node is ready 2022-10-31T23:48:19.073 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com 2022-10-31T23:48:19.132 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102 2022-10-31T23:48:19.500 INFO:teuthology.provision.fog.smithi102:Deploy complete! 2022-10-31T23:48:19.794 INFO:teuthology.lock.ops:Checking smithi043.front.sepia.ceph.com 2022-10-31T23:48:19.796 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-31T23:48:19.820 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com 2022-10-31T23:48:19.821 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-31T23:48:19.842 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com 2022-10-31T23:48:19.843 INFO:teuthology.lock.ops:New key found. Updating... 2022-10-31T23:48:19.865 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: set os type and version on server 2022-10-31T23:48:19.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2022-10-31T23:48:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2022-10-31T23:48:20.195 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2022-10-31T23:48:20.196 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="8" 2022-10-31T23:48:20.196 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2022-10-31T23:48:20.196 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2022-10-31T23:48:20.196 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="8" 2022-10-31T23:48:20.197 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el8" 2022-10-31T23:48:20.197 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-31T23:48:20.197 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2022-10-31T23:48:20.197 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-31T23:48:20.197 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2022-10-31T23:48:20.198 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-31T23:48:20.198 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-31T23:48:20.198 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-31T23:48:20.200 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2022-10-31T23:48:20.222 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server 2022-10-31T23:48:20.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-10-31T23:48:20.429 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2022-10-31T23:48:20.488 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2022-10-31T23:48:20.488 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8" 2022-10-31T23:48:20.488 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2022-10-31T23:48:20.489 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2022-10-31T23:48:20.489 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8" 2022-10-31T23:48:20.489 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2022-10-31T23:48:20.489 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-31T23:48:20.489 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2022-10-31T23:48:20.490 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-31T23:48:20.490 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2022-10-31T23:48:20.490 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-31T23:48:20.490 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-31T23:48:20.491 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-31T23:48:20.491 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-10-31T23:48:20.512 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server 2022-10-31T23:48:20.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-10-31T23:48:20.711 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2022-10-31T23:48:20.770 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2022-10-31T23:48:20.770 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2022-10-31T23:48:20.770 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2022-10-31T23:48:20.771 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2022-10-31T23:48:20.771 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2022-10-31T23:48:20.771 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2022-10-31T23:48:20.771 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2022-10-31T23:48:20.772 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2022-10-31T23:48:20.772 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-10-31T23:48:20.772 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2022-10-31T23:48:20.772 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-10-31T23:48:20.773 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-10-31T23:48:20.773 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-10-31T23:48:20.773 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-10-31T23:48:20.883 INFO:teuthology.dispatcher.supervisor:Running job 7080285 2022-10-31T23:48:20.883 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080285 --name teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} -- /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080285/orig.config.yaml 2022-10-31T23:48:20.889 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080285 2022-10-31T23:48:20.890 INFO:teuthology.dispatcher.supervisor:Job PID: 32401 2022-10-31T23:48:20.890 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-11-01T11:49:09.224 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-11-01T11:49:09.801 INFO:teuthology.kill:Killing Pids: {32401} 2022-11-01T11:49:11.068 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-11-01T11:49:11.069 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['client.0'] 2022-11-01T11:49:11.069 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['client.1'] 2022-11-01T11:49:11.070 INFO:teuthology.misc:Compressing logs... 2022-11-01T11:49:11.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2022-11-01T11:49:11.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-11-01T11:49:11.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-11-01T11:49:11.905 INFO:teuthology.kill:No teuthology processes running 2022-11-01T11:49:13.070 INFO:teuthology.kill:Nuking machines: ['smithi043', 'smithi099', 'smithi102'] 2022-11-01T11:49:14.095 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-11-01T11:49:14.096 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-11-01T11:49:14.097 INFO:teuthology.kill:2022-11-01 11:49:14,095.095 INFO:teuthology.nuke:targets: 2022-11-01T11:49:14.097 INFO:teuthology.kill: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3yy3IgS9xzbwfF3OwBtFnYXoQAVuTtSrCY8lIZSz7+HKhcYXQwF12SdB9JAcUnsjcfu8YALPmyag5ufkYD6hs= 2022-11-01T11:49:14.097 INFO:teuthology.kill: smithi099.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUKMI4KVTlVKwtrTm4rUL4R5TgCR6zBcTmMhY5T3lszA+s8YNbQUT9XZti6zeDjYkQjvj7GsUiNyIZCte+woPykXnGu+hTpV6kUSFl3NJrzkkoAr4N/LBOYtARWsSnS2o0j0/uzrzXp6pgaH0K0RUvoLHoCfcq/T9OqkE3UJ8fyQvwka5R+tRZxg0QRajncCQ9VAJw89Ab2q7HiXK6TU8NBYP8H25sZ/85l/XZkwkAZhxv6FrRVX5sVKNrv/RFRYHsIW0x97zajspWehhmoAGX9OH/wxtNXdcKQ1MzFra1/FtJFGzBr1ioxBFku7qb30aZbhVeR8aquv7UND4DFPU67dnc0dZ0YgE0tGNDqssvxu2bbjA+ODitN610IjzU/hWqQ8P8s57vNt5ylrPgmdGvEiyq9Vt3twxY9LJpM/fB/xWxkPAbvJr8b6hkePKv2GlE6GCf3Scm+BA/D5TfcEvL8FqBrVA3OkcsOo+sbabDFZ9JCiNXlUSzvOIDKXLvnb0= 2022-11-01T11:49:14.098 INFO:teuthology.kill: smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBsSQAhAQO8nanBfIk72h6yFV2DqMje/HXmJv6XQ4Ul+arq5zL/WY7L4+YI4YZVO+xQ7pyohfKR+3TCTTO87VEP5rRH0m3oQ4qI415pIGm/hifjf4G1EVGfbnwzPXnrJAVEixqixkZsOLHvEh/DC20TvuxPguzqI4WdHjUffyssPSCUoyf68ImY1TJoe3LYo4ZD1jyP2b8AUn2+q7AJ5yeeqY3xTAg0ieWJ87aXfRA7Yk51jGZ9UCVLcByQ1vqdaHYAGiXppR7N/FGcJcQs6/26AHlx8Ft6ImiduijLivy9In47N/s26lVfdC2venAWcD10PnNncKfKgK62d/hzsSZuB95xdi9eWP5us1T4YrCW8EdM4rUkKVS0lkBsy6Ucl4KLgjk8iyiyd0yBzoom7IUvBWx1NVh4wda4EESfIm38lwFxO4oIEgY572mbPKOkwA8dq9k91Vb53QkD4h6JI1VD6kNYic8FVlK3H8CqCAHLrDFeiUsG3xYLSO+qGLi578= 2022-11-01T11:49:14.098 INFO:teuthology.kill:2022-11-01 11:49:14,095.095 INFO:teuthology.nuke:Checking targets against current locks 2022-11-01T11:49:14.144 INFO:teuthology.kill:2022-11-01 11:49:14,143.143 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-01T11:49:14.169 INFO:teuthology.kill:2022-11-01 11:49:14,169.169 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-01T11:49:14.200 INFO:teuthology.kill:2022-11-01 11:49:14,200.200 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-01T11:49:14.234 INFO:teuthology.kill:2022-11-01 11:49:14,234.234 INFO:teuthology.orchestra.console:Power off smithi043 2022-11-01T11:49:14.272 INFO:teuthology.kill:2022-11-01 11:49:14,271.271 INFO:teuthology.orchestra.console:Power off smithi099 2022-11-01T11:49:14.311 INFO:teuthology.kill:2022-11-01 11:49:14,311.311 INFO:teuthology.orchestra.console:Power off smithi102 2022-11-01T11:49:22.610 INFO:teuthology.kill:2022-11-01 11:49:22,610.610 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2022-11-01T11:49:22.635 INFO:teuthology.kill:2022-11-01 11:49:22,635.635 INFO:teuthology.orchestra.console:Power off for smithi099 completed 2022-11-01T11:49:22.761 INFO:teuthology.kill:2022-11-01 11:49:22,760.760 INFO:teuthology.lock.ops:unlocked: smithi043.front.sepia.ceph.com 2022-11-01T11:49:22.785 INFO:teuthology.kill:2022-11-01 11:49:22,785.785 INFO:teuthology.lock.ops:unlocked: smithi099.front.sepia.ceph.com 2022-11-01T11:49:26.781 INFO:teuthology.kill:2022-11-01 11:49:26,781.781 INFO:teuthology.orchestra.console:Power off for smithi102 completed 2022-11-01T11:49:26.931 INFO:teuthology.kill:2022-11-01 11:49:26,930.930 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com 2022-11-01T11:51:27.163 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-11-01T11:51:27.302 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi043 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080323 2022-11-01T11:51:27.302 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi099 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080323 2022-11-01T11:51:27.303 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi102 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080324