2021-12-08T20:54:49.542 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-08T20:54:49.676 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi026.front.sepia.ceph.com' reimaging 2021-12-08T20:54:49.677 INFO:teuthology.lock.ops:Updating [ubuntu@smithi026.front.sepia.ceph.com]: reset os type and version on server 2021-12-08T20:54:49.678 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2021-12-08T20:54:49.701 INFO:teuthology.lock.ops:Node 'ubuntu@smithi026.front.sepia.ceph.com' reimaging is complete 2021-12-08T20:54:49.701 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging 2021-12-08T20:54:49.702 INFO:teuthology.lock.ops:Updating [ubuntu@smithi114.front.sepia.ceph.com]: reset os type and version on server 2021-12-08T20:54:49.702 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2021-12-08T20:54:49.724 INFO:teuthology.lock.ops:Node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging is complete 2021-12-08T20:54:49.918 INFO:teuthology.provision.fog.smithi026:Scheduling deploy of centos 8.3 2021-12-08T20:54:49.935 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of centos 8.3 2021-12-08T20:54:50.320 INFO:teuthology.orchestra.console:Power off smithi026 2021-12-08T20:54:50.336 INFO:teuthology.orchestra.console:Power off smithi114 2021-12-08T20:54:58.701 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2021-12-08T20:54:58.803 INFO:teuthology.orchestra.console:Power on smithi026 2021-12-08T20:55:02.825 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2021-12-08T20:55:02.928 INFO:teuthology.orchestra.console:Power on smithi114 2021-12-08T20:55:03.060 INFO:teuthology.orchestra.console:Power on for smithi026 completed 2021-12-08T20:55:03.162 INFO:teuthology.provision.fog.smithi026:Waiting for deploy to finish 2021-12-08T20:55:11.301 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2021-12-08T20:55:11.402 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2021-12-08T20:59:37.840 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-08T20:59:38.135 INFO:teuthology.provision.fog.smithi026:Node is ready 2021-12-08T20:59:38.151 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026.front.sepia.ceph.com 2021-12-08T20:59:38.207 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26 smithi026.front.sepia.ceph.com smithi026 2021-12-08T20:59:38.537 INFO:teuthology.provision.fog.smithi026:Deploy complete! 2021-12-08T21:00:01.200 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-08T21:00:01.491 INFO:teuthology.provision.fog.smithi114:Node is ready 2021-12-08T21:00:01.506 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2021-12-08T21:00:01.561 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2021-12-08T21:00:01.890 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2021-12-08T21:00:02.046 INFO:teuthology.lock.ops:Checking smithi026.front.sepia.ceph.com 2021-12-08T21:00:02.047 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-08T21:00:02.072 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2021-12-08T21:00:02.072 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-08T21:00:02.102 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: set os type and version on server 2021-12-08T21:00:02.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2021-12-08T21:00:02.383 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2021-12-08T21:00:02.439 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Linux" 2021-12-08T21:00:02.440 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8" 2021-12-08T21:00:02.440 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2021-12-08T21:00:02.440 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2021-12-08T21:00:02.440 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8" 2021-12-08T21:00:02.441 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2021-12-08T21:00:02.441 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-08T21:00:02.441 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2021-12-08T21:00:02.441 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-08T21:00:02.442 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2021-12-08T21:00:02.442 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-08T21:00:02.442 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-08T21:00:02.443 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-08T21:00:02.444 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2021-12-08T21:00:02.463 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2021-12-08T21:00:02.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2021-12-08T21:00:02.665 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2021-12-08T21:00:02.720 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Linux" 2021-12-08T21:00:02.721 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8" 2021-12-08T21:00:02.721 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2021-12-08T21:00:02.721 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2021-12-08T21:00:02.721 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8" 2021-12-08T21:00:02.721 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2021-12-08T21:00:02.722 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Linux 8" 2021-12-08T21:00:02.722 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2021-12-08T21:00:02.722 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-08T21:00:02.722 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2021-12-08T21:00:02.723 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-12-08T21:00:02.723 INFO:teuthology.orchestra.run.smithi114.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-12-08T21:00:02.723 INFO:teuthology.orchestra.run.smithi114.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-12-08T21:00:02.724 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2021-12-08T21:00:02.821 INFO:teuthology.dispatcher.supervisor:Running job 6552914 2021-12-08T21:00:02.821 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-12-08_20:49:13-rados-master-distro-basic-smithi/6552914 --name rzarzynski-2021-12-08_20:49:13-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/rzarzynski-2021-12-08_20:49:13-rados-master-distro-basic-smithi/6552914/orig.config.yaml 2021-12-08T21:00:02.825 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-12-08_20:49:13-rados-master-distro-basic-smithi/6552914 2021-12-08T21:00:02.826 INFO:teuthology.dispatcher.supervisor:Job PID: 27856 2021-12-08T21:00:02.827 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-09T09:00:58.146 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-09T09:00:58.363 INFO:teuthology.kill:Killing Pids: {27856} 2021-12-09T09:00:59.552 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-09T09:00:59.553 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-09T09:00:59.553 INFO:teuthology.misc:Compressing logs... 2021-12-09T09:00:59.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2021-12-09T09:00:59.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2021-12-09T09:01:00.180 INFO:teuthology.misc:Compressing logs... 2021-12-09T09:01:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-09T09:01:01.268 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-09T09:01:01.869 INFO:teuthology.misc:Compressing logs... 2021-12-09T09:01:07.261 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /var/log/ceph/ceph-client.0.39648.log: file size changed while zipping 2021-12-09T09:01:22.639 INFO:teuthology.kill:No teuthology processes running 2021-12-09T09:01:23.784 INFO:teuthology.kill:Nuking machines: ['smithi026', 'smithi114'] 2021-12-09T09:01:24.833 INFO:teuthology.kill:2021-12-09 09:01:24,833.833 INFO:teuthology.nuke:targets: 2021-12-09T09:01:24.834 INFO:teuthology.kill: smithi026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC+Rgj0BTZ/WZUxE8ZeSMGJEcyZXJBOtPbnczwkt4ClKbOyez0LuquNfbNuYp3x8+/hzq0JHI3x07MrykaxNxP3PyatosVHki5za9Y0+2UstDHv2/tUdupRtYcNnnPeuxibrox3vMvztFLWiH2v6wzHAqkayiSUSuVQ0RK28ujd4RjWsl15ZsQa+7t33p133PMB/VimtQotQSP9+E+/doInVrXvDS7W/wkxmZUB+Lb5Gah9u5Sk1lwuMnB5qYwip1QUBNe4zjg50+yKquz6dEBn9Mq07AcqASkIlxxiYqzN9lrqvYQvKHQWzN4kFu0/FRVpJkS4RKfAm84EDtlLlcAlc+2aJWXLbn34jrUVvXGA/KRUkFLByn9gq/GURXkU8aV1tamLyR8bm1Ay/9bMBV8B+T52o3ZsBYKp5Ht0QyBDpXSn3X6yzhX6WlLQfeSfDxuTEOhhCT7h2Q/O8MKBwJqvgr4m/8c6imcPtqgwMRJxT+Ygjtymu1ykkFxd25S2ik= 2021-12-09T09:01:24.835 INFO:teuthology.kill: smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKHVuZeiN5/x5ej3TxfxfLgdK9HVn39ygvoE2LmFKPC22lc0/xdarVeABe7khNp0jsoL3olUDQ9QXQyUHLEpK8gkctqVk/+icmPJQqkUn6ZEis8J7BSQPBvRifZ/Er7nMsvek3iXMdra8dTFXm4ljlys8DZFfhv70W/jE8pBWZ1Qvstvc1m+VFecodrdFqUFTGPfmfOt81Z7EZOCeyrNuwNXoD7R4B3MZOC/vzmuTO1LgxRSeSnJS2NtKl0Bvi8Q25kWaJds/XzznIc4OY2PRi5mmwaNS20XqAw49mX9dEJsAHNlCXDSteI0ZE73BhDhbTgFq5e4Kt2uLsSz0fhzCuaCH8ANCmUBp1E5zXLRPoeLBGfO0xiTQurAsaV8baZd7qF15IsBVY7Wnl6WCOwrldAbeRw11NC3L6GMvF0hfz9E6ZFo4TiTZ0EakkyB8d6QnhW7kiYfYWuWtmaox6gCNnZAAdFI1tpitlAlHfZdfOrCQdPbdbcxLSOXREoh4EUzM= 2021-12-09T09:01:24.860 INFO:teuthology.kill:2021-12-09 09:01:24,860.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-09T09:01:24.865 INFO:teuthology.kill:2021-12-09 09:01:24,865.865 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-09T09:01:24.948 INFO:teuthology.kill:2021-12-09 09:01:24,947.947 INFO:teuthology.orchestra.console:Power off smithi114 2021-12-09T09:01:24.962 INFO:teuthology.kill:2021-12-09 09:01:24,961.961 INFO:teuthology.orchestra.console:Power off smithi026 2021-12-09T09:01:33.320 INFO:teuthology.kill:2021-12-09 09:01:33,320.320 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2021-12-09T09:01:33.330 INFO:teuthology.kill:2021-12-09 09:01:33,330.330 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2021-12-09T09:01:33.472 INFO:teuthology.kill:2021-12-09 09:01:33,471.471 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com 2021-12-09T09:01:33.473 INFO:teuthology.kill:2021-12-09 09:01:33,473.473 INFO:teuthology.lock.ops:unlocked: smithi026.front.sepia.ceph.com 2021-12-09T09:03:33.805 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-09T09:03:33.946 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi026 but it was locked by another job: /home/teuthworker/archive/benhanokh-2021-12-08_20:57:58-rados-WIP_GBH_safe_fast_shutdown_6-distro-basic-smithi/6553476 2021-12-09T09:03:33.946 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi114 but it was locked by another job: /home/teuthworker/archive/benhanokh-2021-12-08_20:57:58-rados-WIP_GBH_safe_fast_shutdown_6-distro-basic-smithi/6553476