2021-11-09T14:35:13.614 INFO:root:teuthology version: 1.1.0-27954452 2021-11-09T14:35:13.743 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi013.front.sepia.ceph.com' reimaging 2021-11-09T14:35:13.744 INFO:teuthology.lock.ops:Updating [ubuntu@smithi013.front.sepia.ceph.com]: reset os type and version on server 2021-11-09T14:35:13.745 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2021-11-09T14:35:13.764 INFO:teuthology.lock.ops:Node 'ubuntu@smithi013.front.sepia.ceph.com' reimaging is complete 2021-11-09T14:35:13.765 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging 2021-11-09T14:35:13.765 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server 2021-11-09T14:35:13.766 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-11-09T14:35:13.784 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete 2021-11-09T14:35:13.969 INFO:teuthology.provision.fog.smithi013:Scheduling deploy of rhel 8.4 2021-11-09T14:35:13.990 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of rhel 8.4 2021-11-09T14:35:14.292 INFO:teuthology.orchestra.console:Power off smithi013 2021-11-09T14:35:14.333 INFO:teuthology.orchestra.console:Power off smithi116 2021-11-09T14:35:22.670 INFO:teuthology.orchestra.console:Power off for smithi013 completed 2021-11-09T14:35:22.694 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-11-09T14:35:22.772 INFO:teuthology.orchestra.console:Power on smithi013 2021-11-09T14:35:22.801 INFO:teuthology.orchestra.console:Power on smithi116 2021-11-09T14:35:31.141 INFO:teuthology.orchestra.console:Power on for smithi013 completed 2021-11-09T14:35:31.199 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2021-11-09T14:35:31.243 INFO:teuthology.provision.fog.smithi013:Waiting for deploy to finish 2021-11-09T14:35:31.300 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2021-11-09T14:40:05.694 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-09T14:40:05.912 INFO:teuthology.provision.fog.smithi013:Node is ready 2021-11-09T14:40:05.928 INFO:teuthology.orchestra.run.smithi013.stdout:smithi013.front.sepia.ceph.com 2021-11-09T14:40:05.984 INFO:teuthology.orchestra.run.smithi013.stdout:172.21.15.13 smithi013.front.sepia.ceph.com smithi013 2021-11-09T14:40:06.324 INFO:teuthology.provision.fog.smithi013:Deploy complete! 2021-11-09T14:40:20.715 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-09T14:40:20.968 INFO:teuthology.provision.fog.smithi116:Node is ready 2021-11-09T14:40:20.984 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2021-11-09T14:40:21.041 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2021-11-09T14:40:21.372 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2021-11-09T14:40:21.530 INFO:teuthology.lock.ops:Checking smithi013.front.sepia.ceph.com 2021-11-09T14:40:21.531 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-09T14:40:21.551 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2021-11-09T14:40:21.552 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-09T14:40:21.571 INFO:teuthology.lock.ops:Updating [smithi013.front.sepia.ceph.com]: set os type and version on server 2021-11-09T14:40:21.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi013.front.sepia.ceph.com' 2021-11-09T14:40:21.810 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64 2021-11-09T14:40:21.866 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="Red Hat Enterprise Linux" 2021-11-09T14:40:21.866 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="8.4 (Ootpa)" 2021-11-09T14:40:21.867 INFO:teuthology.orchestra.run.smithi013.stdout:ID="rhel" 2021-11-09T14:40:21.867 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE="fedora" 2021-11-09T14:40:21.867 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="8.4" 2021-11-09T14:40:21.867 INFO:teuthology.orchestra.run.smithi013.stdout:PLATFORM_ID="platform:el8" 2021-11-09T14:40:21.868 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-09T14:40:21.868 INFO:teuthology.orchestra.run.smithi013.stdout:ANSI_COLOR="0;31" 2021-11-09T14:40:21.869 INFO:teuthology.orchestra.run.smithi013.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-09T14:40:21.869 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.redhat.com/" 2021-11-09T14:40:21.869 INFO:teuthology.orchestra.run.smithi013.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-09T14:40:21.869 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-09T14:40:21.870 INFO:teuthology.orchestra.run.smithi013.stdout: 2021-11-09T14:40:21.870 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-09T14:40:21.870 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-09T14:40:21.870 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-09T14:40:21.870 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-09T14:40:21.872 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2021-11-09T14:40:21.891 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2021-11-09T14:40:21.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2021-11-09T14:40:22.076 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2021-11-09T14:40:22.131 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Red Hat Enterprise Linux" 2021-11-09T14:40:22.132 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8.4 (Ootpa)" 2021-11-09T14:40:22.132 INFO:teuthology.orchestra.run.smithi116.stdout:ID="rhel" 2021-11-09T14:40:22.132 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="fedora" 2021-11-09T14:40:22.132 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8.4" 2021-11-09T14:40:22.133 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2021-11-09T14:40:22.133 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-09T14:40:22.133 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2021-11-09T14:40:22.133 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-09T14:40:22.134 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.redhat.com/" 2021-11-09T14:40:22.134 INFO:teuthology.orchestra.run.smithi116.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-09T14:40:22.134 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-09T14:40:22.134 INFO:teuthology.orchestra.run.smithi116.stdout: 2021-11-09T14:40:22.134 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-09T14:40:22.135 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-09T14:40:22.135 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-09T14:40:22.135 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-09T14:40:22.136 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-11-09T14:40:22.238 INFO:teuthology.dispatcher.supervisor:Running job 6492094 2021-11-09T14:40:22.239 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-08_20:18:21-rados-wip-yuri3-testing-2021-11-08-0955-pacific-distro-basic-smithi/6492094 --name yuriw-2021-11-08_20:18:21-rados-wip-yuri3-testing-2021-11-08-0955-pacific-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/yuriw-2021-11-08_20:18:21-rados-wip-yuri3-testing-2021-11-08-0955-pacific-distro-basic-smithi/6492094/orig.config.yaml 2021-11-09T14:40:22.243 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-08_20:18:21-rados-wip-yuri3-testing-2021-11-08-0955-pacific-distro-basic-smithi/6492094 2021-11-09T14:40:22.244 INFO:teuthology.dispatcher.supervisor:Job PID: 29735 2021-11-09T14:40:22.244 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-10T02:40:42.778 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-10T02:40:42.998 INFO:teuthology.kill:Killing Pids: {29735} 2021-11-10T02:40:44.196 INFO:teuthology.task.internal:roles: ubuntu@smithi013.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-11-10T02:40:44.197 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-11-10T02:40:44.197 INFO:teuthology.misc:Compressing logs... 2021-11-10T02:40:44.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi013.front.sepia.ceph.com' 2021-11-10T02:40:44.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2021-11-10T02:40:44.728 INFO:teuthology.misc:Compressing logs... 2021-11-10T02:40:45.854 INFO:teuthology.orchestra.run.smithi116.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-10T02:40:45.939 INFO:teuthology.orchestra.run.smithi013.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-10T02:40:46.600 INFO:teuthology.misc:Compressing logs... 2021-11-10T02:41:53.034 INFO:teuthology.kill:No teuthology processes running 2021-11-10T02:41:54.206 INFO:teuthology.kill:Nuking machines: ['smithi013', 'smithi116'] 2021-11-10T02:41:55.174 INFO:teuthology.kill:2021-11-10 02:41:55,173.173 INFO:teuthology.nuke:targets: 2021-11-10T02:41:55.175 INFO:teuthology.kill: smithi013.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdCgtIuDvaHxuEnyY3bLKaUSMQJN/nqsT7NcUJ8m8Xm9/SbqybV1i8Gf296uf5qwPD4o7pBnGLdOwKDgFN0QWVgnU9dPeZSLUjJ0LamGmEU9vZ00/pda6P1kJLp7ZCrEMB8Twb0Kpkt//3MnHpgK1Wx2DHvy3jCpNtirNnbKtGlCitDwgS5DJxau+O4WKGDcIVqxjvmbEkmdx9OrpY9Vqvv5PTvxQRJf1NSzMBhBrUWCAQlgpZ5sGwic+CovhKpfftXkqcE0LWcDHZKKwBwmlvcrRXfE8fH73JuFO6YtzC6Lum4iKfgcAZtTivs6iSHcvrGL4nQY/g07mG2BJOhZGwdnC4aiwyaRVsYpyZvSrhc9vdjZUXc9LMK/CCOxFTf9WqhQMkJhe1SK0YA9EK6f5DPAmy/iGq6sFNtdZsdLmsjnB8nf0T1IV2OYLqQ9to0BoYuUkpNwymSnBE1FZP63QLL5yZyvq9IA7Lc7OHs2b3IT2GJgeuwu6O+zWDPe3BsZU= 2021-11-10T02:41:55.175 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQtIC/aFgAcMKUbnmz7uGAbc98mtBqp8GIV6ikOYq2eCWvuM0+PiZsqfofDXq8Ssq3+y5Tvl7+RTZ3lQBP52zkv6Krj7QkXTo+5mVAkqumJXOGpV0HvPEqSInQVFE5NZSkpgbkx52mEFsNWYuF+rcoWOPBrtolt6UyRb9LlfAOH9SHsKegrdrW2FNhUjXyB8GebGjouZFPUknwLo53RAj0MPn6L0rkkdFy2sPImIT0o5+VCU7vcw9aHku+2pNBDp2ibLGBOXLZieP4z4Ea1ZgEscf4ZSX+6No3nLLe49nnupx8ywE9bddWUSqC48UEz+OxfR1byBHlxmfl3I7oviRBEd2V6rNDFr2O9O1y0hsEDrGWa9r9sizY/pIxjcq80I3XSVdEQk6XOVR1L3V5IglLOwp7PQNQa+KNJqzKQ4ND0mW9jENWIa6OLNjJZlC6WqG+mJeETNKv6gyVwvsCGfMev3Zt3aGv+X81ffIXh+wCKYtLtOWgFNuqWz1746Danj0= 2021-11-10T02:41:55.203 INFO:teuthology.kill:2021-11-10 02:41:55,203.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-10T02:41:55.207 INFO:teuthology.kill:2021-11-10 02:41:55,207.207 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-10T02:41:55.286 INFO:teuthology.kill:2021-11-10 02:41:55,286.286 INFO:teuthology.orchestra.console:Power off smithi013 2021-11-10T02:41:55.302 INFO:teuthology.kill:2021-11-10 02:41:55,301.301 INFO:teuthology.orchestra.console:Power off smithi116 2021-11-10T02:42:03.657 INFO:teuthology.kill:2021-11-10 02:42:03,657.657 INFO:teuthology.orchestra.console:Power off for smithi013 completed 2021-11-10T02:42:03.665 INFO:teuthology.kill:2021-11-10 02:42:03,665.665 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-11-10T02:42:03.805 INFO:teuthology.kill:2021-11-10 02:42:03,804.804 INFO:teuthology.lock.ops:unlocked: smithi013.front.sepia.ceph.com 2021-11-10T02:42:03.808 INFO:teuthology.kill:2021-11-10 02:42:03,808.808 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2021-11-10T02:44:04.120 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-10T02:44:04.222 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi013 but it was locked by another job: /home/teuthworker/archive/pdonnell-2021-11-09_20:36:56-fs-wip-pdonnell-testing-20211109.180315-distro-basic-smithi/6494407 2021-11-10T02:44:04.223 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi116 but it was locked by another job: /home/teuthworker/archive/pdonnell-2021-11-09_20:36:56-fs-wip-pdonnell-testing-20211109.180315-distro-basic-smithi/6494407