2021-11-04T03:14:27.084 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-04T03:14:27.225 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging 2021-11-04T03:14:27.226 INFO:teuthology.lock.ops:Updating [ubuntu@smithi041.front.sepia.ceph.com]: reset os type and version on server 2021-11-04T03:14:27.226 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-11-04T03:14:27.246 INFO:teuthology.lock.ops:Node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging is complete 2021-11-04T03:14:27.246 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging 2021-11-04T03:14:27.246 INFO:teuthology.lock.ops:Updating [ubuntu@smithi072.front.sepia.ceph.com]: reset os type and version on server 2021-11-04T03:14:27.247 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2021-11-04T03:14:27.267 INFO:teuthology.lock.ops:Node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging is complete 2021-11-04T03:14:27.454 INFO:teuthology.provision.fog.smithi041:Scheduling deploy of rhel 8.4 2021-11-04T03:14:27.485 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of rhel 8.4 2021-11-04T03:14:27.811 INFO:teuthology.orchestra.console:Power off smithi072 2021-11-04T03:14:27.826 INFO:teuthology.orchestra.console:Power off smithi041 2021-11-04T03:14:36.186 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-11-04T03:14:36.288 INFO:teuthology.orchestra.console:Power on smithi041 2021-11-04T03:14:40.295 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2021-11-04T03:14:40.397 INFO:teuthology.orchestra.console:Power on smithi072 2021-11-04T03:14:44.723 INFO:teuthology.orchestra.console:Power on for smithi041 completed 2021-11-04T03:14:44.824 INFO:teuthology.provision.fog.smithi041:Waiting for deploy to finish 2021-11-04T03:14:48.770 INFO:teuthology.orchestra.console:Power on for smithi072 completed 2021-11-04T03:14:48.871 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2021-11-04T03:19:19.220 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-04T03:19:19.441 INFO:teuthology.provision.fog.smithi041:Node is ready 2021-11-04T03:19:19.457 INFO:teuthology.orchestra.run.smithi041.stdout:smithi041.front.sepia.ceph.com 2021-11-04T03:19:19.513 INFO:teuthology.orchestra.run.smithi041.stdout:172.21.15.41 smithi041.front.sepia.ceph.com smithi041 2021-11-04T03:19:19.847 INFO:teuthology.provision.fog.smithi041:Deploy complete! 2021-11-04T03:20:08.546 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-04T03:20:08.780 INFO:teuthology.provision.fog.smithi072:Node is ready 2021-11-04T03:20:08.795 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2021-11-04T03:20:08.850 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2021-11-04T03:20:09.197 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2021-11-04T03:20:09.349 INFO:teuthology.lock.ops:Checking smithi041.front.sepia.ceph.com 2021-11-04T03:20:09.349 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-04T03:20:09.368 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2021-11-04T03:20:09.368 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-04T03:20:09.387 INFO:teuthology.lock.ops:Updating [smithi041.front.sepia.ceph.com]: set os type and version on server 2021-11-04T03:20:09.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi041.front.sepia.ceph.com' 2021-11-04T03:20:09.634 INFO:teuthology.orchestra.run.smithi041.stdout:x86_64 2021-11-04T03:20:09.690 INFO:teuthology.orchestra.run.smithi041.stdout:NAME="Red Hat Enterprise Linux" 2021-11-04T03:20:09.690 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION="8.4 (Ootpa)" 2021-11-04T03:20:09.691 INFO:teuthology.orchestra.run.smithi041.stdout:ID="rhel" 2021-11-04T03:20:09.691 INFO:teuthology.orchestra.run.smithi041.stdout:ID_LIKE="fedora" 2021-11-04T03:20:09.691 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION_ID="8.4" 2021-11-04T03:20:09.691 INFO:teuthology.orchestra.run.smithi041.stdout:PLATFORM_ID="platform:el8" 2021-11-04T03:20:09.692 INFO:teuthology.orchestra.run.smithi041.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-04T03:20:09.692 INFO:teuthology.orchestra.run.smithi041.stdout:ANSI_COLOR="0;31" 2021-11-04T03:20:09.692 INFO:teuthology.orchestra.run.smithi041.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-04T03:20:09.692 INFO:teuthology.orchestra.run.smithi041.stdout:HOME_URL="https://www.redhat.com/" 2021-11-04T03:20:09.692 INFO:teuthology.orchestra.run.smithi041.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-04T03:20:09.693 INFO:teuthology.orchestra.run.smithi041.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-04T03:20:09.693 INFO:teuthology.orchestra.run.smithi041.stdout: 2021-11-04T03:20:09.693 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-04T03:20:09.693 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-04T03:20:09.694 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-04T03:20:09.694 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-04T03:20:09.695 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-11-04T03:20:09.714 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2021-11-04T03:20:09.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2021-11-04T03:20:09.903 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2021-11-04T03:20:09.959 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Red Hat Enterprise Linux" 2021-11-04T03:20:09.959 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8.4 (Ootpa)" 2021-11-04T03:20:09.959 INFO:teuthology.orchestra.run.smithi072.stdout:ID="rhel" 2021-11-04T03:20:09.960 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="fedora" 2021-11-04T03:20:09.960 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8.4" 2021-11-04T03:20:09.960 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2021-11-04T03:20:09.960 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-11-04T03:20:09.961 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2021-11-04T03:20:09.961 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-11-04T03:20:09.961 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.redhat.com/" 2021-11-04T03:20:09.961 INFO:teuthology.orchestra.run.smithi072.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-11-04T03:20:09.961 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-04T03:20:09.962 INFO:teuthology.orchestra.run.smithi072.stdout: 2021-11-04T03:20:09.962 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-04T03:20:09.962 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-11-04T03:20:09.962 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-11-04T03:20:09.963 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-11-04T03:20:09.963 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2021-11-04T03:20:10.064 INFO:teuthology.dispatcher.supervisor:Running job 6482411 2021-11-04T03:20:10.065 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482411 --name yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects} -- /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482411/orig.config.yaml 2021-11-04T03:20:10.068 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482411 2021-11-04T03:20:10.069 INFO:teuthology.dispatcher.supervisor:Job PID: 27194 2021-11-04T03:20:10.069 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-04T15:20:30.502 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-04T15:20:30.710 INFO:teuthology.kill:Killing Pids: {27194} 2021-11-04T15:20:31.915 INFO:teuthology.task.internal:roles: ubuntu@smithi041.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-11-04T15:20:31.916 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-11-04T15:20:31.916 INFO:teuthology.misc:Compressing logs... 2021-11-04T15:20:31.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi041.front.sepia.ceph.com' 2021-11-04T15:20:32.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2021-11-04T15:20:32.506 INFO:teuthology.misc:Compressing logs... 2021-11-04T15:20:33.634 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-04T15:20:33.698 INFO:teuthology.orchestra.run.smithi041.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-04T15:21:07.633 INFO:teuthology.misc:Compressing logs... 2021-11-04T15:21:50.705 INFO:teuthology.kill:No teuthology processes running 2021-11-04T15:21:51.918 INFO:teuthology.kill:Nuking machines: ['smithi041', 'smithi072'] 2021-11-04T15:21:52.899 INFO:teuthology.kill:2021-11-04 15:21:52,899.899 INFO:teuthology.nuke:targets: 2021-11-04T15:21:52.900 INFO:teuthology.kill: smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyfeLjAVt6g+74NoaOrcIcqQaTk3zaiHliMbM2nzP+LG+pKKQaDxcY+aYz8TSXhhJKZmNrRfpsS7yaba8ABa9ipFuEvqh+Rnf+/KvT4feTTP/h8xvD9+snawiXMh2judaiKx/t2UM6e2Jaoct7fkpXNPV3oncUzlujU278QW7rGu48bq1LTRAi6boyTdSnN20SA4CViyR6ol/0Azl3GLOkOiM2gbOkwgnKXluXIBif7T7QhCg1vcvSpvy0qoAg/mq8f2Qk6fVS/I6CHXH7w4gdPnx1fW4PoCv49iZd2GiZ3n+rfYSQvLr132ng96xXYPhdxA+e56dEcIS4FgECS99XyGYE7C7FctpnM1BbSGbAjQk6U6v97AlrKWPhI7F7OzwbyXpOCqcan9eYNIf5v+ZxHmTax5ADxZ8xZn5HRFjL0j10nI7MFyMF6c/gggahC7UMSH64ALk6zcVO6PcOjJS+9pPJMIXSzIt0615+e3mMPMJ9736WJUK21MExBTUtAE8= 2021-11-04T15:21:52.900 INFO:teuthology.kill: smithi072.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR0CmRqGbKdNpXPZrmInYevUUDq6YqSuCwH5UvzdPP63ZQ+wwnnH3XZMt0AbDBmNgRClYbB4poDloHJIyxDtzjLgUHtihnI8UrbDmNGBQXtDgmOV8ulzGmxNHFjg5zDByIiv+ZnpFiHMWfO3v8/fqGm7NChB/DrfAHG2qE6nDuuvineMikbUIIyaH68XdSrQnbK0fKqb3eICDxoaw+y+Wp1YraAih16kM2qttrhequz3QhK10NKQswoRtlt1Fx7nwt27VmEsmbFpm4PKzbFBVaqVZu9bIDGZknzJ1ukFT+dJjH9Vww8IxQMWQ2+1u3EQKQakvt6KJMWMWNKlNavyYgc657MZCJFQCEkbjQ3yTYNer30KetoZ1UBDdi0AJ1qKMuRpA5Oo1qLkEo5rfT+F58q3ZKimYfKnZ7IBEpmPpJRy48VYHi1MNhZWS1jVVqY9rXS3mdy77+82WUhBjaI6gKaVSWnRpbCqA8kXvzZG5uAMIqBXvZvNZ6qDfweEvITIU= 2021-11-04T15:21:52.925 INFO:teuthology.kill:2021-11-04 15:21:52,925.925 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-04T15:21:52.935 INFO:teuthology.kill:2021-11-04 15:21:52,935.935 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-04T15:21:53.009 INFO:teuthology.kill:2021-11-04 15:21:53,008.008 INFO:teuthology.orchestra.console:Power off smithi072 2021-11-04T15:21:53.022 INFO:teuthology.kill:2021-11-04 15:21:53,021.021 INFO:teuthology.orchestra.console:Power off smithi041 2021-11-04T15:22:01.377 INFO:teuthology.kill:2021-11-04 15:22:01,377.377 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2021-11-04T15:22:01.549 INFO:teuthology.kill:2021-11-04 15:22:01,548.548 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2021-11-04T15:22:05.496 INFO:teuthology.kill:2021-11-04 15:22:05,495.495 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-11-04T15:22:05.642 INFO:teuthology.kill:2021-11-04 15:22:05,642.642 INFO:teuthology.lock.ops:unlocked: smithi041.front.sepia.ceph.com 2021-11-04T15:24:05.970 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-04T15:24:06.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi041 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-02_19:49:55-rados-wip-yuri8-testing-2021-11-02-1009-pacific-distro-basic-smithi/6479119 2021-11-04T15:24:06.084 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi072 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-02_19:49:55-rados-wip-yuri8-testing-2021-11-02-1009-pacific-distro-basic-smithi/6479119