2022-08-17T08:25:21.079 INFO:root:teuthology version: 1.1.1.dev412+gb1d387f1 2022-08-17T08:25:21.285 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2022-08-17T08:25:21.287 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2022-08-17T08:25:21.287 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2022-08-17T08:25:21.311 INFO:teuthology.lock.ops:Node 'smithi132.front.sepia.ceph.com' reimaging is complete 2022-08-17T08:25:21.311 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging 2022-08-17T08:25:21.312 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server 2022-08-17T08:25:21.312 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-08-17T08:25:21.347 INFO:teuthology.lock.ops:Node 'smithi161.front.sepia.ceph.com' reimaging is complete 2022-08-17T08:25:21.348 INFO:teuthology.lock.ops:Start node 'smithi202.front.sepia.ceph.com' reimaging 2022-08-17T08:25:21.348 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: reset os type and version on server 2022-08-17T08:25:21.349 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2022-08-17T08:25:21.371 INFO:teuthology.lock.ops:Node 'smithi202.front.sepia.ceph.com' reimaging is complete 2022-08-17T08:25:21.536 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 8.stream 2022-08-17T08:25:21.568 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 8.stream 2022-08-17T08:25:21.596 INFO:teuthology.provision.fog.smithi202:Scheduling deploy of centos 8.stream 2022-08-17T08:25:21.844 INFO:teuthology.orchestra.console:Power off smithi132 2022-08-17T08:25:21.865 INFO:teuthology.orchestra.console:Power off smithi161 2022-08-17T08:25:21.925 INFO:teuthology.orchestra.console:Power off smithi202 2022-08-17T08:25:30.237 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2022-08-17T08:25:30.244 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-08-17T08:25:30.339 INFO:teuthology.orchestra.console:Power on smithi132 2022-08-17T08:25:30.347 INFO:teuthology.orchestra.console:Power on smithi161 2022-08-17T08:25:34.403 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2022-08-17T08:25:34.505 INFO:teuthology.orchestra.console:Power on smithi202 2022-08-17T08:25:38.707 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2022-08-17T08:25:38.713 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2022-08-17T08:25:38.809 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2022-08-17T08:25:38.815 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2022-08-17T08:25:42.867 INFO:teuthology.orchestra.console:Power on for smithi202 completed 2022-08-17T08:25:42.968 INFO:teuthology.provision.fog.smithi202:Waiting for deploy to finish 2022-08-17T08:29:42.316 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-17T08:30:06.337 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-17T08:30:08.000 INFO:teuthology.provision.fog.smithi132:Node is ready 2022-08-17T08:30:08.024 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2022-08-17T08:30:08.084 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2022-08-17T08:30:08.453 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2022-08-17T08:30:20.975 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-17T08:30:21.269 INFO:teuthology.provision.fog.smithi202:Node is ready 2022-08-17T08:30:21.287 INFO:teuthology.orchestra.run.smithi202.stdout:smithi202.front.sepia.ceph.com 2022-08-17T08:30:21.347 INFO:teuthology.orchestra.run.smithi202.stdout:172.21.15.202 smithi202.front.sepia.ceph.com smithi202 2022-08-17T08:30:21.719 INFO:teuthology.provision.fog.smithi202:Deploy complete! 2022-08-17T08:30:32.069 INFO:teuthology.provision.fog.smithi161:Node is ready 2022-08-17T08:30:32.090 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2022-08-17T08:30:32.150 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2022-08-17T08:30:32.524 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2022-08-17T08:30:32.833 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2022-08-17T08:30:32.835 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-17T08:30:32.856 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com 2022-08-17T08:30:32.857 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-17T08:30:32.877 INFO:teuthology.lock.ops:Checking smithi202.front.sepia.ceph.com 2022-08-17T08:30:32.877 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-17T08:30:32.898 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2022-08-17T08:30:32.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2022-08-17T08:30:33.158 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2022-08-17T08:30:33.215 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2022-08-17T08:30:33.215 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8" 2022-08-17T08:30:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2022-08-17T08:30:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2022-08-17T08:30:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8" 2022-08-17T08:30:33.217 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2022-08-17T08:30:33.217 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-17T08:30:33.217 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2022-08-17T08:30:33.217 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-17T08:30:33.218 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2022-08-17T08:30:33.218 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-17T08:30:33.218 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-17T08:30:33.218 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-17T08:30:33.220 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2022-08-17T08:30:33.240 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server 2022-08-17T08:30:33.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-08-17T08:30:33.442 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2022-08-17T08:30:33.500 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2022-08-17T08:30:33.501 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8" 2022-08-17T08:30:33.501 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2022-08-17T08:30:33.502 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2022-08-17T08:30:33.502 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8" 2022-08-17T08:30:33.503 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2022-08-17T08:30:33.503 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-17T08:30:33.504 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2022-08-17T08:30:33.504 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-17T08:30:33.504 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2022-08-17T08:30:33.505 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-17T08:30:33.505 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-17T08:30:33.506 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-17T08:30:33.506 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-08-17T08:30:33.527 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: set os type and version on server 2022-08-17T08:30:33.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2022-08-17T08:30:33.763 INFO:teuthology.orchestra.run.smithi202.stdout:x86_64 2022-08-17T08:30:33.822 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="CentOS Stream" 2022-08-17T08:30:33.822 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8" 2022-08-17T08:30:33.823 INFO:teuthology.orchestra.run.smithi202.stdout:ID="centos" 2022-08-17T08:30:33.823 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="rhel fedora" 2022-08-17T08:30:33.823 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8" 2022-08-17T08:30:33.823 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2022-08-17T08:30:33.823 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-17T08:30:33.824 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2022-08-17T08:30:33.824 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-17T08:30:33.824 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://centos.org/" 2022-08-17T08:30:33.824 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-17T08:30:33.825 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-17T08:30:33.825 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-17T08:30:33.825 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2022-08-17T08:30:34.030 INFO:teuthology.dispatcher.supervisor:Running job 6976512 2022-08-17T08:30:34.030 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-08-17_08:06:32-fs-wip-vshankar-testing1-20220817-114751-testing-default-smithi/6976512 --name vshankar-2022-08-17_08:06:32-fs-wip-vshankar-testing1-20220817-114751-testing-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} -- /home/teuthworker/archive/vshankar-2022-08-17_08:06:32-fs-wip-vshankar-testing1-20220817-114751-testing-default-smithi/6976512/orig.config.yaml 2022-08-17T08:30:34.035 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-08-17_08:06:32-fs-wip-vshankar-testing1-20220817-114751-testing-default-smithi/6976512 2022-08-17T08:30:34.036 INFO:teuthology.dispatcher.supervisor:Job PID: 30710 2022-08-17T08:30:34.036 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-08-17T20:31:29.855 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-08-17T20:31:30.062 INFO:teuthology.kill:Killing Pids: {30710} 2022-08-17T20:31:31.510 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-08-17T20:31:31.511 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['client.0'] 2022-08-17T20:31:31.511 INFO:teuthology.task.internal:roles: ubuntu@smithi202.front.sepia.ceph.com - ['client.1'] 2022-08-17T20:31:31.512 INFO:teuthology.misc:Compressing logs... 2022-08-17T20:31:31.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2022-08-17T20:31:31.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-08-17T20:31:32.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2022-08-17T20:31:32.882 INFO:teuthology.misc:Compressing logs... 2022-08-17T20:32:07.346 INFO:teuthology.misc:Compressing logs... 2022-08-17T20:38:24.743 INFO:teuthology.kill:No teuthology processes running 2022-08-17T20:38:26.126 INFO:teuthology.kill:Nuking machines: ['smithi132', 'smithi161', 'smithi202'] 2022-08-17T20:38:27.234 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-08-17T20:38:27.235 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-08-17T20:38:27.236 INFO:teuthology.kill:2022-08-17 20:38:27,234.234 INFO:teuthology.nuke:targets: 2022-08-17T20:38:27.236 INFO:teuthology.kill: smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxHRphJyzEkwvQl9nrbfjcPaMcCt60zN+IY2nE2dD4Azx+31l8luGr0dBBbbZQhXaGthCVyBkkZY5drtotuNRG54/6UXLBM4lqeNrN/MQkVxKuR2cZRgndwW0q6T4lAw2oUoLsF7xYR7MA2mhpV3lxtWGkZigQQ4qRof4tYepFsXAlh+ws4c44E1O7oSSe0FoHUUNLnX+ydmAagCzHR/nuCWM5dDXE0i0v/71w+bgyPECfvoP7+gwVR4Ibpc1mLc0JExdCxoHOt1XFkqc4zGLGVWhqKzDW3oRz4Wmfe50mgDj65JwWAEDNz12+zUGxY43vEUdaBIxqkh1peXfEVEJwPvGzRTYTOVvYPk6683dSYZuS0Mf0y11XxY8KPOWDtdT1R1l3A/wng/sniQRVa6XBRDtmeEwX/37Vo7paqUHkG1KCP57w05/axfhwsYs9YDP3bi+KtdjohK44LuKUdoQ6IdvYbeADXk5cOJKavRYVcGVAU8/33xPzoHhazBZ6hXE= 2022-08-17T20:38:27.236 INFO:teuthology.kill: smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPoxAaOCySvN4EIBnmUlVybAYSJGe+x7tM0Q806dyXf8TAhFVQNM1GWosAfRv4Rt7namxpsZFNDQgR/Ci9jnG6UNm3sqwWxs87M9WPgJZFnqpcvfsFIaQAFV2r8jdJYYAfbIpwGNh4e2j6ZrUTfpELh05LBZTKoq5Zlc2Is36yb45qLJIk2Nf+cQnKCbN8BTEOevpQPdD4nEw1xfInycDuXQlkTWWLe/A5r0uxsPyF3MgI0gu74sjcjXle1De4yXjZ3KqYOsRBMdT3IB1BNER+TQJ0KVttKIMYOBwyFc+p5LDeeh1Mbl6DsEyHQj0Axa8W220ZSGVZBhi2lkk4RmHgjQMBKlDe7Jt124wjsxRiCJcZNyY5ksBKMbc0lP5xAVLdpUQW7ksa5fEvo7TDBBDTiMu4yQIoL/jYPutvQ3kj9iXaS36piwA1h9XOzUdJJ9gG4dG/KSfGYlZgarnL30v07/QMmiEYmls2Cp6nPL1JHizzvFkEpseFELrTt27XRik= 2022-08-17T20:38:27.237 INFO:teuthology.kill: smithi202.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8wAdmlKFg0W3c0ZvnY4UEz5O0iy88eSqRGmqpt2Zx1GZoadppB1QuMh66B3ksGH4oCcr4pOwEMyU3Nye1iuHLBdr0xkKJBHAMToXnIuNQ6T/GOBf9HLD3Oi3ey77iFegvX+47JXPnil3FAC4P3BfqU9yamYNk8vE2k0jAQb8u3p5CNPRwXaavgespQWELWuzz14WM4WkBV3w+FWuBEtfiwQ43HL43cJd6Qdla4F5lz8iDUO3bM1XOGDaHLvks9e8U+i2OREtPvUocx/38JTnHC9f8FwH4SeVMxVapIYfFGskAUgKi3q5jlqEB0ak/OzGq/N1TqlRCg2U9owPigN6LTn+UqYBExJL8wUSHHtywbjhte9uy/xSWEuAYLQEO9QmODcSAKCj7NnQAYoEFjwJxSvmTwhahAdwxGJrgJO6ZqfjLLN0o1g49c0eY9zVWfaSD8V1bNS4TFicIpe4CCrFxwmnxuLjcsafic+nVF9Iva0xgwc1TDpXGEPD0J1k61L0= 2022-08-17T20:38:27.237 INFO:teuthology.kill:2022-08-17 20:38:27,234.234 INFO:teuthology.nuke:Checking targets against current locks 2022-08-17T20:38:27.295 INFO:teuthology.kill:2022-08-17 20:38:27,294.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-17T20:38:27.322 INFO:teuthology.kill:2022-08-17 20:38:27,322.322 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-17T20:38:27.337 INFO:teuthology.kill:2022-08-17 20:38:27,336.336 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-17T20:38:27.381 INFO:teuthology.kill:2022-08-17 20:38:27,381.381 INFO:teuthology.orchestra.console:Power off smithi132 2022-08-17T20:38:27.426 INFO:teuthology.kill:2022-08-17 20:38:27,425.425 INFO:teuthology.orchestra.console:Power off smithi161 2022-08-17T20:38:27.442 INFO:teuthology.kill:2022-08-17 20:38:27,441.441 INFO:teuthology.orchestra.console:Power off smithi202 2022-08-17T20:38:35.773 INFO:teuthology.kill:2022-08-17 20:38:35,773.773 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2022-08-17T20:38:35.796 INFO:teuthology.kill:2022-08-17 20:38:35,796.796 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-08-17T20:38:35.810 INFO:teuthology.kill:2022-08-17 20:38:35,810.810 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2022-08-17T20:38:35.933 INFO:teuthology.kill:2022-08-17 20:38:35,932.932 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com 2022-08-17T20:38:35.945 INFO:teuthology.kill:2022-08-17 20:38:35,945.945 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com 2022-08-17T20:38:35.957 INFO:teuthology.kill:2022-08-17 20:38:35,957.957 INFO:teuthology.lock.ops:unlocked: smithi202.front.sepia.ceph.com 2022-08-17T20:40:36.321 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-08-17T20:40:36.516 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi161 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-08-17_18:46:04-fs-wip-yuri7-testing-2022-08-17-0943-quincy-distro-default-smithi/6977234 2022-08-17T20:40:36.517 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi202 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-08-17_18:46:04-fs-wip-yuri7-testing-2022-08-17-0943-quincy-distro-default-smithi/6977234