2022-03-01T22:08:09.797 INFO:root:teuthology version: 1.1.0-cb5b1a75 2022-03-01T22:08:09.926 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi081.front.sepia.ceph.com' reimaging 2022-03-01T22:08:09.928 INFO:teuthology.lock.ops:Updating [ubuntu@smithi081.front.sepia.ceph.com]: reset os type and version on server 2022-03-01T22:08:09.928 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2022-03-01T22:08:09.946 INFO:teuthology.lock.ops:Node 'ubuntu@smithi081.front.sepia.ceph.com' reimaging is complete 2022-03-01T22:08:09.947 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging 2022-03-01T22:08:09.947 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server 2022-03-01T22:08:09.947 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-03-01T22:08:09.965 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete 2022-03-01T22:08:10.167 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of rhel 8.4 2022-03-01T22:08:10.171 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of rhel 8.4 2022-03-01T22:08:10.491 INFO:teuthology.orchestra.console:Power off smithi092 2022-03-01T22:08:10.530 INFO:teuthology.orchestra.console:Power off smithi081 2022-03-01T22:08:22.977 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-03-01T22:08:23.001 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2022-03-01T22:08:23.079 INFO:teuthology.orchestra.console:Power on smithi092 2022-03-01T22:08:23.103 INFO:teuthology.orchestra.console:Power on smithi081 2022-03-01T22:08:27.352 INFO:teuthology.orchestra.console:Power on for smithi081 completed 2022-03-01T22:08:27.453 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish 2022-03-01T22:08:31.456 INFO:teuthology.orchestra.console:Power on for smithi092 completed 2022-03-01T22:08:31.557 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2022-03-01T22:12:37.755 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-01T22:12:58.267 INFO:teuthology.provision.fog.smithi092:Node is ready 2022-03-01T22:12:58.287 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2022-03-01T22:12:58.344 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2022-03-01T22:12:58.684 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2022-03-01T22:13:08.123 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-01T22:13:08.350 INFO:teuthology.provision.fog.smithi081:Node is ready 2022-03-01T22:13:08.365 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081.front.sepia.ceph.com 2022-03-01T22:13:08.420 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081.front.sepia.ceph.com smithi081 2022-03-01T22:13:08.735 INFO:teuthology.provision.fog.smithi081:Deploy complete! 2022-03-01T22:13:08.884 INFO:teuthology.lock.ops:Checking smithi081.front.sepia.ceph.com 2022-03-01T22:13:08.884 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-01T22:13:08.905 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2022-03-01T22:13:08.905 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-01T22:13:08.922 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: set os type and version on server 2022-03-01T22:13:08.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2022-03-01T22:13:09.146 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2022-03-01T22:13:09.200 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Red Hat Enterprise Linux" 2022-03-01T22:13:09.201 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="8.4 (Ootpa)" 2022-03-01T22:13:09.201 INFO:teuthology.orchestra.run.smithi081.stdout:ID="rhel" 2022-03-01T22:13:09.201 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="fedora" 2022-03-01T22:13:09.201 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="8.4" 2022-03-01T22:13:09.201 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el8" 2022-03-01T22:13:09.202 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-01T22:13:09.202 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2022-03-01T22:13:09.202 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-01T22:13:09.202 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.redhat.com/" 2022-03-01T22:13:09.203 INFO:teuthology.orchestra.run.smithi081.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-01T22:13:09.203 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-01T22:13:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: 2022-03-01T22:13:09.203 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-01T22:13:09.203 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-01T22:13:09.204 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-01T22:13:09.204 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-01T22:13:09.205 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2022-03-01T22:13:09.223 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2022-03-01T22:13:09.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-03-01T22:13:09.497 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2022-03-01T22:13:09.551 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2022-03-01T22:13:09.551 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.4 (Ootpa)" 2022-03-01T22:13:09.552 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2022-03-01T22:13:09.552 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2022-03-01T22:13:09.552 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.4" 2022-03-01T22:13:09.552 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2022-03-01T22:13:09.553 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-01T22:13:09.553 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2022-03-01T22:13:09.553 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-01T22:13:09.553 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2022-03-01T22:13:09.553 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-01T22:13:09.554 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-01T22:13:09.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2022-03-01T22:13:09.554 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-01T22:13:09.554 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-01T22:13:09.555 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-01T22:13:09.555 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-01T22:13:09.555 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2022-03-01T22:13:09.644 INFO:teuthology.dispatcher.supervisor:Running job 6715291 2022-03-01T22:13:09.645 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_cb5b1a75970b7aa81700a0ab081943b3146d40d0/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2022-03-01_21:56:54-rados-master-distro-basic-smithi/6715291 --name rzarzynski-2022-03-01_21:56:54-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/rzarzynski-2022-03-01_21:56:54-rados-master-distro-basic-smithi/6715291/orig.config.yaml 2022-03-01T22:13:09.648 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2022-03-01_21:56:54-rados-master-distro-basic-smithi/6715291 2022-03-01T22:13:09.649 INFO:teuthology.dispatcher.supervisor:Job PID: 22811 2022-03-01T22:13:09.649 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-02T04:43:39.633 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-02T04:43:39.789 INFO:teuthology.kill:Killing Pids: {22811} 2022-03-02T04:43:41.079 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-03-02T04:43:41.080 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-03-02T04:43:41.080 INFO:teuthology.misc:Compressing logs... 2022-03-02T04:43:41.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2022-03-02T04:43:41.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2022-03-02T04:43:41.611 INFO:teuthology.misc:Compressing logs... 2022-03-02T04:43:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-03-02T04:43:42.237 INFO:teuthology.orchestra.run.smithi081.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-03-02T04:43:42.589 INFO:teuthology.misc:Compressing logs... 2022-03-02T04:43:49.642 INFO:teuthology.kill:No teuthology processes running 2022-03-02T04:43:50.796 INFO:teuthology.kill:Nuking machines: ['smithi081', 'smithi092'] 2022-03-02T04:43:51.751 INFO:teuthology.kill:2022-03-02 04:43:51,750.750 INFO:teuthology.nuke:targets: 2022-03-02T04:43:51.752 INFO:teuthology.kill: smithi081.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDLH0v5KZEDlAlfYVmKSZeDopxjj9CyvCe0TuRivn2MDCGxMyAzbN/YBK9xBrdHrE8iEkqCzOQOB0LigmDnHbjxBuVp5QG6jEKNgUHJiy/gqoUp2s/2DZjeYy3VcrF48F4dF+UKKtW67ICI44ntRymg6b0j/l9sJw387K5SclF/uvlkmeaV6IdlMpPqjEGJIo2p0GqFjI/TjclhE2aRyoIUT3aGn3O2fzmEAunUcC1UXqM4MedzzAdc8PJToE4bCoAp1gK+kE6hUUj8HG0MCjZBX9jGvjVKikL/iB0raKwZuTJzbCpkFQQSEdHFRJbdcsZ9c+i2HqRbmWJSkudkAKiMUakPLB3svKGg36JSw7QDqZWllbxED5/JARJzjSwL2rYkdJaVQwe8urcoWFM9p8Rg9p6fiPVcxL0cR0a0Urpw+MDesLhYMJLBHiB1+Ns7EpBMb8//Fer2HPfHYFYjbIP4bz+YfvPa9P2di2uQEbpLINMy7JQfe0y4Tao/eIju3s= 2022-03-02T04:43:51.752 INFO:teuthology.kill: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1YPusxZ9aPaQTRawcUH5DWTCV9fj+hU787IE/uVL5+pXH7NKn74yzYEGnqqMD4na5DAvR0xaBzyvhdCQiZStN7SN9+mKbVwlGZeLUw4bFm5dxRF1JY8fFBsEJBmcmCdffMEuhAu4n3CuEZTTjc6p8iLBm8jtZiNWnotU14Yp1ZBRWEfN4q2kfE/VNXnNi1343qKItLovDl2qKIyUk1vy2kRLkuxVGNHLCNACbwfAU9NOImv0FP6tNkrlef8BtFBTDXeJgCUip9tZh5l6ABOagd2V30I3K/mSLaLfiiL0gcpAAqbGcR6kd7HV1MEPAz6YsmA/zfhTcAqEsAWj6vt35/WdMSI4JTIJR6qPw6rFM4wu5Cr22R5W5KwNs4uK0iwHkw0cUwItT5lONMJ8zdE++QGg69ieNouUjgZs/+BRH7s/4Kt+JTsA0jBvgQBB8YDHnBqt4XRdea3dfwgXOWkH13ZjcL/gZR7cDJONmeBkUQ7ij8T0vLiuWOB1geHfl/v0= 2022-03-02T04:43:51.777 INFO:teuthology.kill:2022-03-02 04:43:51,777.777 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-02T04:43:51.780 INFO:teuthology.kill:2022-03-02 04:43:51,780.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-02T04:43:51.855 INFO:teuthology.kill:2022-03-02 04:43:51,854.854 INFO:teuthology.orchestra.console:Power off smithi092 2022-03-02T04:43:51.869 INFO:teuthology.kill:2022-03-02 04:43:51,868.868 INFO:teuthology.orchestra.console:Power off smithi081 2022-03-02T04:44:00.231 INFO:teuthology.kill:2022-03-02 04:44:00,230.230 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2022-03-02T04:44:00.377 INFO:teuthology.kill:2022-03-02 04:44:00,376.376 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com 2022-03-02T04:44:04.336 INFO:teuthology.kill:2022-03-02 04:44:04,336.336 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2022-03-02T04:44:04.488 INFO:teuthology.kill:2022-03-02 04:44:04,488.488 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com 2022-03-02T04:46:04.761 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-02T04:46:04.869 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi081 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-01_17:45:51-rados-wip-yuri3-testing-2022-02-28-0757-pacific-distro-default-smithi/6714959 2022-03-02T04:46:04.869 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi092 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-01_17:45:51-rados-wip-yuri3-testing-2022-02-28-0757-pacific-distro-default-smithi/6714959