2021-11-13T19:17:43.588 INFO:root:teuthology version: 1.1.0-d4737010 2021-11-13T19:17:43.724 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging 2021-11-13T19:17:43.725 INFO:teuthology.lock.ops:Updating [ubuntu@smithi107.front.sepia.ceph.com]: reset os type and version on server 2021-11-13T19:17:43.725 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-11-13T19:17:43.746 INFO:teuthology.lock.ops:Node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging is complete 2021-11-13T19:17:43.746 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging 2021-11-13T19:17:43.746 INFO:teuthology.lock.ops:Updating [ubuntu@smithi138.front.sepia.ceph.com]: reset os type and version on server 2021-11-13T19:17:43.747 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2021-11-13T19:17:43.766 INFO:teuthology.lock.ops:Node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging is complete 2021-11-13T19:17:43.961 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of centos 8.stream 2021-11-13T19:17:44.031 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.stream 2021-11-13T19:17:44.281 INFO:teuthology.orchestra.console:Power off smithi107 2021-11-13T19:17:44.326 INFO:teuthology.orchestra.console:Power off smithi138 2021-11-13T19:17:56.777 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-11-13T19:17:56.802 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2021-11-13T19:17:56.879 INFO:teuthology.orchestra.console:Power on smithi107 2021-11-13T19:17:56.903 INFO:teuthology.orchestra.console:Power on smithi138 2021-11-13T19:18:01.154 INFO:teuthology.orchestra.console:Power on for smithi107 completed 2021-11-13T19:18:01.161 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2021-11-13T19:18:01.256 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2021-11-13T19:18:01.262 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2021-11-13T19:22:50.816 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-13T19:22:51.108 INFO:teuthology.provision.fog.smithi138:Node is ready 2021-11-13T19:22:51.126 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2021-11-13T19:22:51.184 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2021-11-13T19:22:51.536 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2021-11-13T19:23:02.683 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-13T19:23:02.967 INFO:teuthology.provision.fog.smithi107:Node is ready 2021-11-13T19:23:02.983 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2021-11-13T19:23:03.040 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2021-11-13T19:23:03.382 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2021-11-13T19:23:03.576 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2021-11-13T19:23:03.576 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-13T19:23:03.597 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2021-11-13T19:23:03.597 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-13T19:23:03.618 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2021-11-13T19:23:03.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2021-11-13T19:23:03.849 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2021-11-13T19:23:03.927 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2021-11-13T19:23:04.677 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8" 2021-11-13T19:23:04.678 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2021-11-13T19:23:04.678 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2021-11-13T19:23:04.679 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8" 2021-11-13T19:23:04.679 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2021-11-13T19:23:04.679 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-13T19:23:04.679 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2021-11-13T19:23:04.680 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-13T19:23:04.680 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2021-11-13T19:23:04.681 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-13T19:23:04.681 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-13T19:23:04.681 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-13T19:23:04.682 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-11-13T19:23:04.704 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2021-11-13T19:23:04.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2021-11-13T19:23:05.024 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2021-11-13T19:23:05.088 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2021-11-13T19:23:05.089 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2021-11-13T19:23:05.089 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2021-11-13T19:23:05.089 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2021-11-13T19:23:05.089 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2021-11-13T19:23:05.090 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2021-11-13T19:23:05.090 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-13T19:23:05.090 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2021-11-13T19:23:05.090 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-13T19:23:05.091 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2021-11-13T19:23:05.091 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-13T19:23:05.091 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-13T19:23:05.091 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-13T19:23:05.092 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2021-11-13T19:23:05.227 INFO:teuthology.dispatcher.supervisor:Running job 6501938 2021-11-13T19:23:05.228 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d4737010a85099043cf081dc05b4069d301b23fb/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501938 --name yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501938/orig.config.yaml 2021-11-13T19:23:05.231 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-13_17:22:34-rados-wip-yuri6-testing-2021-11-12-0808-distro-basic-smithi/6501938 2021-11-13T19:23:05.232 INFO:teuthology.dispatcher.supervisor:Job PID: 30775 2021-11-13T19:23:05.232 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-14T07:24:00.380 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-14T07:24:00.591 INFO:teuthology.kill:Killing Pids: {30775} 2021-11-14T07:24:01.761 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2021-11-14T07:24:01.762 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2021-11-14T07:24:01.762 INFO:teuthology.misc:Compressing logs... 2021-11-14T07:24:01.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2021-11-14T07:24:01.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2021-11-14T07:24:02.270 INFO:teuthology.misc:Compressing logs... 2021-11-14T07:24:02.592 INFO:teuthology.misc:Compressing logs... 2021-11-14T07:24:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-mon.smithi138.log: file size changed while zipping 2021-11-14T07:24:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-11-14T07:24:08.842 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2021-11-14T07:24:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-11-14T07:24:12.729 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-mon.smithi107.log: file size changed while zipping 2021-11-14T07:24:15.791 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-11-14T07:24:16.352 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-11-14T07:24:18.943 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-mgr.smithi107.bfqtvf.log: file size changed while zipping 2021-11-14T07:24:23.289 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/0a61d9e2-44b9-11ec-8c2c-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-11-14T07:24:44.266 INFO:teuthology.kill:No teuthology processes running 2021-11-14T07:24:45.427 INFO:teuthology.kill:Nuking machines: ['smithi107', 'smithi138'] 2021-11-14T07:24:46.503 INFO:teuthology.kill:2021-11-14 07:24:46,502.502 INFO:teuthology.nuke:targets: 2021-11-14T07:24:46.505 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDha5J8EVBPDLomp7oGxse5CoGANV74Bu1gKSvTWjkIkILNfWQWf0HlF8byXvdwVe5FG0zd67N89MML7wF2siK3r3cVHuxHK3ZzEaNJOLHhYvY+ut0S4jGa5eplTbykk9JbTTZlDOYnsMgNgESqF4OWnvU9ZdDmFnMClKRCyfVhhw9EaHmL0j+s5jmLXh4K+Z1PSstR0epFE6nr0P3whoBfcS4cyZxjyjZGzlvSNas1cSNdwdDq/DhxILph3Cvdz2XdIsZBUQoNwI118W9A6tljrqi3BTbxrXzghnrXXjsJC0c+D1quOpXSIrrA7GnnDvufhjvsRrMtwtuYLhGp3+KirjPMRfulBtbEU9ZTbnxNmwKrwnqBY1tMghApj49AzRDOOIljm4SG1UUot/EcOSinnvs1A//4Tc50H3Ci6rgE7stRUrhx71egvW4e/lI3yo8KBMwpjAYu0LlMOt3+m0Pps6Hf8KwrI7t6SpIU6z3qFpjWoNm5oFT6LyWXQeTQnR0= 2021-11-14T07:24:46.506 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0AHO6jZ747mvee/t+RgEIcfs0K9070ZCg/3W3ehg7uk9Pcpla/+QN5I5VzaTH4MqFngHAtTOxgka69uZfP+fuHHhfgG4xRqlE4HELNeNPJ7Pd+q369mH/1Ge0Ok692QrBnkT8zaeqw3uWeYlLfZpvUzW2mPScsCKxZs2+RQBquSF/w+ymhn0Kf7CFbT1rVPNDH1iCeVB7ytZYU5N5KH1bT63xqU/MuA2peZgT9LmMB6XrQ319cy2Rr1UP2FVNpsnlYyi6Hh5hz2wcbzXK5jCJfcGbdujzUl03jk9XYOv7qBGjjp/6IpxYge/vRX0BcnQgOZJGt0sFn+lv/InL7k8YiStnRPX4kUXlFilAbVcgw20/vbndpPqaqqgc7O0gZ5ffc0QdfVCjgnXaooQE+n5jTY18ma5peUgQZ+hkBdiAYfiO6EXobZUATYFhNcL+9Uy9V1f8Z+xjNh7yZ9clL0DV1k7kWz+LSNyFpAs7Fy+iPIefckdWS+iBjWK6f6HtbQ0= 2021-11-14T07:24:46.543 INFO:teuthology.kill:2021-11-14 07:24:46,542.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-14T07:24:46.546 INFO:teuthology.kill:2021-11-14 07:24:46,546.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-14T07:24:46.633 INFO:teuthology.kill:2021-11-14 07:24:46,633.633 INFO:teuthology.orchestra.console:Power off smithi107 2021-11-14T07:24:46.648 INFO:teuthology.kill:2021-11-14 07:24:46,648.648 INFO:teuthology.orchestra.console:Power off smithi138 2021-11-14T07:24:55.080 INFO:teuthology.kill:2021-11-14 07:24:55,080.080 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2021-11-14T07:24:55.229 INFO:teuthology.kill:2021-11-14 07:24:55,228.228 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2021-11-14T07:24:59.168 INFO:teuthology.kill:2021-11-14 07:24:59,167.167 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-11-14T07:24:59.318 INFO:teuthology.kill:2021-11-14 07:24:59,318.318 INFO:teuthology.lock.ops:unlocked: smithi107.front.sepia.ceph.com 2021-11-14T07:26:59.617 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15