2022-08-14T19:05:54.027 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-14T19:05:54.212 INFO:teuthology.lock.ops:Start node 'smithi029.front.sepia.ceph.com' reimaging 2022-08-14T19:05:54.213 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: reset os type and version on server 2022-08-14T19:05:54.213 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-08-14T19:05:54.235 INFO:teuthology.lock.ops:Node 'smithi029.front.sepia.ceph.com' reimaging is complete 2022-08-14T19:05:54.235 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging 2022-08-14T19:05:54.235 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server 2022-08-14T19:05:54.236 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2022-08-14T19:05:54.255 INFO:teuthology.lock.ops:Node 'smithi071.front.sepia.ceph.com' reimaging is complete 2022-08-14T19:05:54.255 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging 2022-08-14T19:05:54.256 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server 2022-08-14T19:05:54.256 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-08-14T19:05:54.276 INFO:teuthology.lock.ops:Node 'smithi161.front.sepia.ceph.com' reimaging is complete 2022-08-14T19:05:54.442 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of centos 8.stream 2022-08-14T19:05:54.456 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of centos 8.stream 2022-08-14T19:05:54.476 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 8.stream 2022-08-14T19:05:54.759 INFO:teuthology.orchestra.console:Power off smithi029 2022-08-14T19:05:54.776 INFO:teuthology.orchestra.console:Power off smithi071 2022-08-14T19:05:54.812 INFO:teuthology.orchestra.console:Power off smithi161 2022-08-14T19:05:54.920 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2022-08-14T19:05:54.949 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-08-14T19:05:55.022 INFO:teuthology.orchestra.console:Power on smithi071 2022-08-14T19:05:55.050 INFO:teuthology.orchestra.console:Power on smithi161 2022-08-14T19:06:03.131 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-08-14T19:06:03.233 INFO:teuthology.orchestra.console:Power on smithi029 2022-08-14T19:06:03.382 INFO:teuthology.orchestra.console:Power on for smithi071 completed 2022-08-14T19:06:03.408 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2022-08-14T19:06:03.483 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2022-08-14T19:06:03.510 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2022-08-14T19:06:11.716 INFO:teuthology.orchestra.console:Power on for smithi029 completed 2022-08-14T19:06:11.817 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2022-08-14T19:10:32.414 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-14T19:10:32.673 INFO:teuthology.provision.fog.smithi071:Node is ready 2022-08-14T19:10:32.691 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2022-08-14T19:10:32.750 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2022-08-14T19:10:33.112 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2022-08-14T19:10:46.345 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-14T19:10:55.708 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-08-14T19:10:56.072 INFO:teuthology.provision.fog.smithi029:Node is ready 2022-08-14T19:10:56.094 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2022-08-14T19:10:56.154 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2022-08-14T19:10:56.555 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2022-08-14T19:11:11.324 INFO:teuthology.provision.fog.smithi161:Node is ready 2022-08-14T19:11:11.346 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2022-08-14T19:11:11.406 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2022-08-14T19:11:11.782 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2022-08-14T19:11:12.011 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2022-08-14T19:11:12.013 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-14T19:11:12.037 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2022-08-14T19:11:12.038 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-14T19:11:12.057 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com 2022-08-14T19:11:12.057 INFO:teuthology.lock.ops:New key found. Updating... 2022-08-14T19:11:12.079 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2022-08-14T19:11:12.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-08-14T19:11:12.352 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2022-08-14T19:11:12.416 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2022-08-14T19:11:12.416 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8" 2022-08-14T19:11:12.416 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2022-08-14T19:11:12.417 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2022-08-14T19:11:12.417 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8" 2022-08-14T19:11:12.417 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2022-08-14T19:11:12.417 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-14T19:11:12.418 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2022-08-14T19:11:12.418 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-14T19:11:12.418 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2022-08-14T19:11:12.418 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-14T19:11:12.419 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-14T19:11:12.419 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-14T19:11:12.421 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-08-14T19:11:12.443 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2022-08-14T19:11:12.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2022-08-14T19:11:12.711 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2022-08-14T19:11:12.769 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2022-08-14T19:11:12.769 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2022-08-14T19:11:12.770 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2022-08-14T19:11:12.770 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2022-08-14T19:11:12.770 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2022-08-14T19:11:12.770 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2022-08-14T19:11:12.771 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-14T19:11:12.771 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2022-08-14T19:11:12.771 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-14T19:11:12.771 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2022-08-14T19:11:12.772 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-14T19:11:12.772 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-14T19:11:12.772 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-14T19:11:12.773 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2022-08-14T19:11:12.794 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server 2022-08-14T19:11:12.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-08-14T19:11:13.011 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2022-08-14T19:11:13.069 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2022-08-14T19:11:13.069 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8" 2022-08-14T19:11:13.070 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2022-08-14T19:11:13.070 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2022-08-14T19:11:13.073 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8" 2022-08-14T19:11:13.073 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2022-08-14T19:11:13.073 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 8" 2022-08-14T19:11:13.073 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2022-08-14T19:11:13.074 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-08-14T19:11:13.074 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2022-08-14T19:11:13.074 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-08-14T19:11:13.075 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-08-14T19:11:13.075 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-08-14T19:11:13.075 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2022-08-14T19:11:13.179 INFO:teuthology.dispatcher.supervisor:Running job 6969340 2022-08-14T19:11:13.179 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969340 --name vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} -- /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969340/orig.config.yaml 2022-08-14T19:11:13.184 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969340 2022-08-14T19:11:13.185 INFO:teuthology.dispatcher.supervisor:Job PID: 3120 2022-08-14T19:11:13.186 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-08-15T07:12:08.887 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-08-15T07:12:09.079 INFO:teuthology.kill:Killing Pids: {3120} 2022-08-15T07:12:10.436 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-08-15T07:12:10.437 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['client.0'] 2022-08-15T07:12:10.437 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['client.1'] 2022-08-15T07:12:10.438 INFO:teuthology.misc:Compressing logs... 2022-08-15T07:12:10.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-08-15T07:12:10.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2022-08-15T07:12:10.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2022-08-15T07:12:11.706 INFO:teuthology.misc:Compressing logs... 2022-08-15T07:12:47.004 INFO:teuthology.misc:Compressing logs... 2022-08-15T07:14:40.750 INFO:teuthology.kill:No teuthology processes running 2022-08-15T07:14:42.162 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi071', 'smithi161'] 2022-08-15T07:14:43.278 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_9e7483cc68a9eb6b54dacbb0bec3bf23a5d32425/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-08-15T07:14:43.279 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-08-15T07:14:43.280 INFO:teuthology.kill:2022-08-15 07:14:43,278.278 INFO:teuthology.nuke:targets: 2022-08-15T07:14:43.280 INFO:teuthology.kill: smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDyGJZDlIppBLGHn7/KG5OIclj20LkLF6NqaQeCsLn7rmYdnsOL+NUIOwTn5yw9xQfX793mV6oA8r4xMpxd/F0sbac1B+fsBjiCpxD95pKRgbbCyuyhCWQseamsmI2O7LOKBBi1+W+fVDJP2SCxyyuJmf31h6ukJFF7zxliaWdoHypN2PgIi5xvUlfq7jwqojEX1g2aVzeilKGphMXZN4s39tIn407t2VWLWCSSwg7xIcC6VZFYr05iPzvmBlIaHgfRGugbZiWHo+B6Ag+qS0X49DS/yXayF0ZEVJ2qZ2e3MMgvWBGyOCZiwfJyPI8hX4YGINllP1F/tFKXcyXoDPJLH4z+qkad3RUeAjlvae12/0OJTPNS66ENEouxeFdi64yV9je3GGstKlIr7AWH5htcKWkpXLtBJEizrOV7ZV43SDQlbl8fgRyE6qnxH2y1J3n5aDoakcKhgLK/Dt5lXxQ2Nh2tewdO0olKj30sayMT0Dej0fvApc3MPNbcY7fLK80= 2022-08-15T07:14:43.280 INFO:teuthology.kill: smithi071.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGYyom3cm/bse1J9SzsSrOO4v2uYllezcBzK9UWzF+i5dTM9TbT3WXdkBkCSXAgNg/18HzRqu3eXLjZVUmE+fD5Fazd1SuX906cBiBrxk0FDP5UFReHswvPz/L3jTL6k7Jlqw6SbuxpMnQdJKbttsxcxXDJN8yGyawWXemiNB32YVtEE5Q0zDKV18Fn/TmL6dOLbaPMbPkiomEoML/4+UO00bnRn+mk+zT7rewvfHf1MNul4AalcR68wwl+eh9XmnoyVuuRmndKxY3/nTY4aUc9yeRG8tHtmCRkkFSUm5nftFUzm/JBF7ya4byPrNDDzE2l4GPpyRdmS1QS2DkHedBbRDvMyAEsbWsVIBqsiv3EPdYWKtZb38qqZtTnw0qQcUEziJUViUn4Q5SuLQsrgT2CdVtq4C82kcFvQNtVZqGQde+e9JFZCqwiu4eLfS1MXhzSyj6/pI6WcAMWEpK4T1H3clR3GglMooBJNtIE8VxAWK2Fgro1HNsDP/ScQX1Ol0= 2022-08-15T07:14:43.280 INFO:teuthology.kill: smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDV3MCzYLs8XJN9x3WshBuBHlkVs/ZcEBpf0MZRgwPb21v3QbXQmaKNDhRY9d8/98KqPPtEVW9OjQbEA6PXA1BDx0tZAiYFLwTvq7r3H5jwEm/EMVOOkCKVDLKRlcAfj2foZv6aCbqgLk6RbfmieArGMbLmOKucU2GywV89uY/azH5M8C5Ay2mipAYQO70qU6++kTlDm/0c1CNTvfFaLAN+/MJp4oM+2Uo9+b/xOjisPcHbCVB91znN8JN72guoL7rm6rz77vakQnxe1vx2rW0zGZKUdnuKfN/UcmQd4NePP6jDckpLLR3woyqRESMp1Zjhi8oKPo1DBZ7+UisxMwPphbFoS1FZxffgK7J1BGs55oH47c0QLpPzRfLMkwx+VM3nn0Vnp+olGO9M5a5pYw5vMiPexcgfrVXb6USI+8VfQ7MkckFGqFEvsA4NOvBfnomJup+ffBEZFP/Q9jpw+mkhNuhkLgMJmouofU5O9shtPqUlJhViTl494mdEy6o+P+U= 2022-08-15T07:14:43.281 INFO:teuthology.kill:2022-08-15 07:14:43,278.278 INFO:teuthology.nuke:Checking targets against current locks 2022-08-15T07:14:43.335 INFO:teuthology.kill:2022-08-15 07:14:43,335.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-15T07:14:43.359 INFO:teuthology.kill:2022-08-15 07:14:43,359.359 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-15T07:14:43.384 INFO:teuthology.kill:2022-08-15 07:14:43,383.383 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-15T07:14:43.423 INFO:teuthology.kill:2022-08-15 07:14:43,423.423 INFO:teuthology.orchestra.console:Power off smithi029 2022-08-15T07:14:43.459 INFO:teuthology.kill:2022-08-15 07:14:43,458.458 INFO:teuthology.orchestra.console:Power off smithi071 2022-08-15T07:14:43.501 INFO:teuthology.kill:2022-08-15 07:14:43,501.501 INFO:teuthology.orchestra.console:Power off smithi161 2022-08-15T07:14:51.792 INFO:teuthology.kill:2022-08-15 07:14:51,791.791 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-08-15T07:14:51.860 INFO:teuthology.kill:2022-08-15 07:14:51,860.860 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2022-08-15T07:14:51.951 INFO:teuthology.kill:2022-08-15 07:14:51,950.950 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2022-08-15T07:14:52.010 INFO:teuthology.kill:2022-08-15 07:14:52,010.010 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com 2022-08-15T07:14:55.938 INFO:teuthology.kill:2022-08-15 07:14:55,938.938 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2022-08-15T07:14:56.085 INFO:teuthology.kill:2022-08-15 07:14:56,085.085 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com 2022-08-15T07:16:56.365 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-08-15T07:16:56.508 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi071 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969471 2022-08-15T07:16:56.509 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi161 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/6969471