2021-09-18T19:37:34.528 INFO:root:teuthology version: 1.1.0-4e1f59de 2021-09-18T19:37:34.676 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging 2021-09-18T19:37:34.678 INFO:teuthology.lock.ops:Updating [ubuntu@smithi165.front.sepia.ceph.com]: reset os type and version on server 2021-09-18T19:37:34.678 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2021-09-18T19:37:34.705 INFO:teuthology.lock.ops:Node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging is complete 2021-09-18T19:37:34.706 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging 2021-09-18T19:37:34.706 INFO:teuthology.lock.ops:Updating [ubuntu@smithi198.front.sepia.ceph.com]: reset os type and version on server 2021-09-18T19:37:34.707 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server 2021-09-18T19:37:34.726 INFO:teuthology.lock.ops:Node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging is complete 2021-09-18T19:37:34.905 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of rhel 8.4 2021-09-18T19:37:34.916 INFO:teuthology.provision.fog.smithi198:Scheduling deploy of rhel 8.4 2021-09-18T19:37:35.263 INFO:teuthology.orchestra.console:Power off smithi165 2021-09-18T19:37:35.288 INFO:teuthology.orchestra.console:Power off smithi198 2021-09-18T19:37:43.644 INFO:teuthology.orchestra.console:Power off for smithi165 completed 2021-09-18T19:37:43.745 INFO:teuthology.orchestra.console:Power on smithi165 2021-09-18T19:37:47.761 INFO:teuthology.orchestra.console:Power off for smithi198 completed 2021-09-18T19:37:47.863 INFO:teuthology.orchestra.console:Power on smithi198 2021-09-18T19:37:47.994 INFO:teuthology.orchestra.console:Power on for smithi165 completed 2021-09-18T19:37:48.095 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish 2021-09-18T19:37:56.225 INFO:teuthology.orchestra.console:Power on for smithi198 completed 2021-09-18T19:37:56.326 INFO:teuthology.provision.fog.smithi198:Waiting for deploy to finish 2021-09-18T19:42:28.119 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-18T19:42:28.330 INFO:teuthology.provision.fog.smithi198:Node is ready 2021-09-18T19:42:28.347 INFO:teuthology.orchestra.run.smithi198.stdout:smithi198.front.sepia.ceph.com 2021-09-18T19:42:28.404 INFO:teuthology.orchestra.run.smithi198.stdout:172.21.15.198 smithi198.front.sepia.ceph.com smithi198 2021-09-18T19:42:28.739 INFO:teuthology.provision.fog.smithi198:Deploy complete! 2021-09-18T19:42:44.080 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-18T19:42:44.290 INFO:teuthology.provision.fog.smithi165:Node is ready 2021-09-18T19:42:44.305 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com 2021-09-18T19:42:44.361 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165 2021-09-18T19:42:44.674 INFO:teuthology.provision.fog.smithi165:Deploy complete! 2021-09-18T19:42:44.810 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com 2021-09-18T19:42:44.811 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-18T19:42:44.830 INFO:teuthology.lock.ops:Checking smithi198.front.sepia.ceph.com 2021-09-18T19:42:44.830 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-18T19:42:44.852 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server 2021-09-18T19:42:44.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2021-09-18T19:42:45.074 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2021-09-18T19:42:45.129 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="Red Hat Enterprise Linux" 2021-09-18T19:42:45.129 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8.4 (Ootpa)" 2021-09-18T19:42:45.129 INFO:teuthology.orchestra.run.smithi165.stdout:ID="rhel" 2021-09-18T19:42:45.129 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="fedora" 2021-09-18T19:42:45.130 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8.4" 2021-09-18T19:42:45.130 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8" 2021-09-18T19:42:45.130 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-09-18T19:42:45.130 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2021-09-18T19:42:45.131 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-09-18T19:42:45.131 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://www.redhat.com/" 2021-09-18T19:42:45.131 INFO:teuthology.orchestra.run.smithi165.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-09-18T19:42:45.131 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-18T19:42:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: 2021-09-18T19:42:45.132 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-18T19:42:45.132 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-09-18T19:42:45.132 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-09-18T19:42:45.132 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-09-18T19:42:45.133 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2021-09-18T19:42:45.150 INFO:teuthology.lock.ops:Updating [smithi198.front.sepia.ceph.com]: set os type and version on server 2021-09-18T19:42:45.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi198.front.sepia.ceph.com' 2021-09-18T19:42:45.384 INFO:teuthology.orchestra.run.smithi198.stdout:x86_64 2021-09-18T19:42:45.440 INFO:teuthology.orchestra.run.smithi198.stdout:NAME="Red Hat Enterprise Linux" 2021-09-18T19:42:45.440 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION="8.4 (Ootpa)" 2021-09-18T19:42:45.440 INFO:teuthology.orchestra.run.smithi198.stdout:ID="rhel" 2021-09-18T19:42:45.441 INFO:teuthology.orchestra.run.smithi198.stdout:ID_LIKE="fedora" 2021-09-18T19:42:45.441 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION_ID="8.4" 2021-09-18T19:42:45.441 INFO:teuthology.orchestra.run.smithi198.stdout:PLATFORM_ID="platform:el8" 2021-09-18T19:42:45.441 INFO:teuthology.orchestra.run.smithi198.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-09-18T19:42:45.442 INFO:teuthology.orchestra.run.smithi198.stdout:ANSI_COLOR="0;31" 2021-09-18T19:42:45.442 INFO:teuthology.orchestra.run.smithi198.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-09-18T19:42:45.442 INFO:teuthology.orchestra.run.smithi198.stdout:HOME_URL="https://www.redhat.com/" 2021-09-18T19:42:45.442 INFO:teuthology.orchestra.run.smithi198.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-09-18T19:42:45.442 INFO:teuthology.orchestra.run.smithi198.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-18T19:42:45.443 INFO:teuthology.orchestra.run.smithi198.stdout: 2021-09-18T19:42:45.443 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-18T19:42:45.443 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-09-18T19:42:45.443 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-09-18T19:42:45.444 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-09-18T19:42:45.444 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server 2021-09-18T19:42:45.534 INFO:teuthology.dispatcher.supervisor:Running job 6396351 2021-09-18T19:42:45.534 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_4e1f59de82924f116bd6b91664f90ef632c60a3f/virtualenv/bin/teuthology -v --owner scheduled_ozeneva@teuthology --archive /home/teuthworker/archive/ozeneva-2021-09-18_18:19:17-rados-wip-omri-tracer-2-distro-basic-smithi/6396351 --name ozeneva-2021-09-18_18:19:17-rados-wip-omri-tracer-2-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} -- /home/teuthworker/archive/ozeneva-2021-09-18_18:19:17-rados-wip-omri-tracer-2-distro-basic-smithi/6396351/orig.config.yaml 2021-09-18T19:42:45.538 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ozeneva-2021-09-18_18:19:17-rados-wip-omri-tracer-2-distro-basic-smithi/6396351 2021-09-18T19:42:45.540 INFO:teuthology.dispatcher.supervisor:Job PID: 2349 2021-09-18T19:42:45.540 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-09-19T07:43:37.395 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-09-19T07:43:37.587 INFO:teuthology.kill:Killing Pids: {2349} 2021-09-19T07:43:38.796 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-09-19T07:43:38.797 INFO:teuthology.task.internal:roles: ubuntu@smithi198.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-09-19T07:43:38.798 INFO:teuthology.misc:Compressing logs... 2021-09-19T07:43:38.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2021-09-19T07:43:38.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi198.front.sepia.ceph.com' 2021-09-19T07:43:39.405 INFO:teuthology.misc:Compressing logs... 2021-09-19T07:43:40.527 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-19T07:43:40.528 INFO:teuthology.orchestra.run.smithi198.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-09-19T07:44:24.162 INFO:teuthology.misc:Compressing logs... 2021-09-19T07:44:28.057 INFO:teuthology.kill:No teuthology processes running 2021-09-19T07:44:29.214 INFO:teuthology.kill:Nuking machines: ['smithi165', 'smithi198'] 2021-09-19T07:44:30.240 INFO:teuthology.kill:2021-09-19 07:44:30,240.240 INFO:teuthology.nuke:targets: 2021-09-19T07:44:30.241 INFO:teuthology.kill: smithi165.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC42cFFA2V9mKVXNQUNkuHkyWpdQpgHju5MWbqZf9zpWhdUA/vWgXg/3Fw/PXSCg5pkB2Ou6w7RcsgfEfUuwhR6dL72mZ4N1HNnG4DrzcfX/Blm0V5XfS0yIX3BsyCMYDqCrlrRhLqWTa7stGSkW3/4GuZc6ugiBg3CZRkaohddUHPIvBxGWFxNd92sO8ugnsHjOX/ZorK9tvcttLNIxYS+P7MaIL1KBJiJO1wZ/2WvYL05Jz3BOOCB/HFadqD9sIvAjYvBqta3TL1Tlj4H5zcvXJwo29femTTGls8qVtv1sNXWYIiKWgmuvLB04H/FM7INNRKtyYhHFbwonuqBUfKVKNPlWv7rf43EfmKXs6vlyIBs0953Q0dwdo21VctTKqdep8m+GTeCq0kb4PAtBii32xm3Bmm+HgM98ggoqhr1ZcOUWb07G+5KfYBnxkRpls62+UzdZTzK/ZhlumLL3rkCNNXSs8kZBvPMcHeJPlA2cgedH9DHs2lmmJh9lSY3ZRc= 2021-09-19T07:44:30.241 INFO:teuthology.kill: smithi198.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe1fjLnlMtgQ5q9jUEWr48J542CtRw7HzmjwS6ussYjbOYyIjx2y3nYl4YGPAUsTjViNh3sAWVtjz3PKeWZ4HEQYcUCumBnurT9O2yqvuQBBJVVwCkQvz5KLuybb6xj5RztXktOYnc/SXZinhJWrLJtMQcv/JCJHbzh/WdbShkWT8vpWl/r/zwtAMZHRvu9r/DzP//t5xGHMpG8GG2+Zo/uCKburykdmR0sFZ9MOLS4glWlH0cRm51oE77fYYbMqaOz20RKCdZ8DPluZdKD61vqMaOOKWuKOS9FA/Jm1mGuH8ebFR+JdvLVqTouSMFy0GPZKpgnZvDenXXV8ZGAbm0TtTGTrMaBCZ5RKuZTFcXIa0tlzScpo9iwUqncxc7CStkJrFmS+YvoAcOmRfsfOF3ifmnxruUlSs0oYtk6qbhmzRzEj6x8lu6xN9mhnvu0ohaNLatyBmeXCebP+uYDsX1GAlitLpg/QKaYIAA1xudKG5eMAw+Un6Ps/J+oNZMnTM= 2021-09-19T07:44:30.262 INFO:teuthology.kill:2021-09-19 07:44:30,262.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-19T07:44:30.265 INFO:teuthology.kill:2021-09-19 07:44:30,265.265 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-19T07:44:30.321 INFO:teuthology.kill:2021-09-19 07:44:30,321.321 INFO:teuthology.orchestra.console:Power off smithi165 2021-09-19T07:44:30.335 INFO:teuthology.kill:2021-09-19 07:44:30,334.334 INFO:teuthology.orchestra.console:Power off smithi198 2021-09-19T07:44:42.804 INFO:teuthology.kill:2021-09-19 07:44:42,803.803 INFO:teuthology.orchestra.console:Power off for smithi165 completed 2021-09-19T07:44:42.810 INFO:teuthology.kill:2021-09-19 07:44:42,810.810 INFO:teuthology.orchestra.console:Power off for smithi198 completed 2021-09-19T07:44:42.952 INFO:teuthology.kill:2021-09-19 07:44:42,951.951 INFO:teuthology.lock.ops:unlocked smithi165.front.sepia.ceph.com 2021-09-19T07:44:42.955 INFO:teuthology.kill:2021-09-19 07:44:42,955.955 INFO:teuthology.lock.ops:unlocked smithi198.front.sepia.ceph.com 2021-09-19T07:46:43.256 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-09-19T07:46:43.344 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-09-19T07:46:43.345 INFO:teuthology.nuke:Checking targets against current locks 2021-09-19T07:46:43.454 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-19T07:46:43.458 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-19T07:46:43.519 INFO:teuthology.orchestra.console:Power off smithi198 2021-09-19T07:46:43.535 INFO:teuthology.orchestra.console:Power off smithi165 2021-09-19T07:46:51.892 INFO:teuthology.orchestra.console:Power off for smithi198 completed 2021-09-19T07:46:51.898 INFO:teuthology.orchestra.console:Power off for smithi165 completed 2021-09-19T07:46:52.046 INFO:teuthology.lock.ops:unlocked smithi198.front.sepia.ceph.com 2021-09-19T07:46:52.050 INFO:teuthology.lock.ops:unlocked smithi165.front.sepia.ceph.com