2022-02-06T18:39:41.784 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-06T18:39:41.910 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging 2022-02-06T18:39:41.911 INFO:teuthology.lock.ops:Updating [ubuntu@smithi178.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T18:39:41.911 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2022-02-06T18:39:41.930 INFO:teuthology.lock.ops:Node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging is complete 2022-02-06T18:39:41.931 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging 2022-02-06T18:39:41.931 INFO:teuthology.lock.ops:Updating [ubuntu@smithi198.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T18:39:41.932 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server 2022-02-06T18:39:41.951 INFO:teuthology.lock.ops:Node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging is complete 2022-02-06T18:39:42.162 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of ubuntu 20.04 2022-02-06T18:39:42.205 INFO:teuthology.provision.fog.smithi198:Scheduling deploy of ubuntu 20.04 2022-02-06T18:39:42.474 INFO:teuthology.orchestra.console:Power off smithi178 2022-02-06T18:39:42.518 INFO:teuthology.orchestra.console:Power off smithi198 2022-02-06T18:39:50.856 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2022-02-06T18:39:50.958 INFO:teuthology.orchestra.console:Power on smithi178 2022-02-06T18:39:55.005 INFO:teuthology.orchestra.console:Power off for smithi198 completed 2022-02-06T18:39:55.107 INFO:teuthology.orchestra.console:Power on smithi198 2022-02-06T18:39:59.329 INFO:teuthology.orchestra.console:Power on for smithi178 completed 2022-02-06T18:39:59.431 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2022-02-06T18:40:03.479 INFO:teuthology.orchestra.console:Power on for smithi198 completed 2022-02-06T18:40:03.582 INFO:teuthology.provision.fog.smithi198:Waiting for deploy to finish 2022-02-06T18:45:00.714 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T18:45:01.174 INFO:teuthology.provision.fog.smithi198:Node is ready 2022-02-06T18:45:01.181 INFO:teuthology.orchestra.run.smithi198.stdout:smithi198.front.sepia.ceph.com 2022-02-06T18:45:01.230 INFO:teuthology.orchestra.run.smithi198.stdout:172.21.15.198 smithi198.front.sepia.ceph.com smithi198 2022-02-06T18:45:01.396 INFO:teuthology.provision.fog.smithi198:Deploy complete! 2022-02-06T18:45:23.666 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T18:45:24.127 INFO:teuthology.provision.fog.smithi178:Node is ready 2022-02-06T18:45:24.133 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com 2022-02-06T18:45:24.141 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2022-02-06T18:45:24.339 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2022-02-06T18:45:24.525 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2022-02-06T18:45:24.526 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T18:45:24.549 INFO:teuthology.lock.ops:Checking smithi198.front.sepia.ceph.com 2022-02-06T18:45:24.550 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T18:45:24.569 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2022-02-06T18:45:24.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2022-02-06T18:45:24.734 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2022-02-06T18:45:24.741 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2022-02-06T18:45:24.742 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T18:45:24.742 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2022-02-06T18:45:24.742 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2022-02-06T18:45:24.743 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T18:45:24.743 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="20.04" 2022-02-06T18:45:24.743 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T18:45:24.743 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T18:45:24.743 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T18:45:24.744 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T18:45:24.744 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=focal 2022-02-06T18:45:24.744 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=focal 2022-02-06T18:45:24.745 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2022-02-06T18:45:24.762 INFO:teuthology.lock.ops:Updating [smithi198.front.sepia.ceph.com]: set os type and version on server 2022-02-06T18:45:24.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi198.front.sepia.ceph.com' 2022-02-06T18:45:24.951 INFO:teuthology.orchestra.run.smithi198.stdout:x86_64 2022-02-06T18:45:24.957 INFO:teuthology.orchestra.run.smithi198.stdout:NAME="Ubuntu" 2022-02-06T18:45:24.958 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T18:45:24.958 INFO:teuthology.orchestra.run.smithi198.stdout:ID=ubuntu 2022-02-06T18:45:24.958 INFO:teuthology.orchestra.run.smithi198.stdout:ID_LIKE=debian 2022-02-06T18:45:24.958 INFO:teuthology.orchestra.run.smithi198.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T18:45:24.959 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION_ID="20.04" 2022-02-06T18:45:24.959 INFO:teuthology.orchestra.run.smithi198.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T18:45:24.959 INFO:teuthology.orchestra.run.smithi198.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T18:45:24.959 INFO:teuthology.orchestra.run.smithi198.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T18:45:24.959 INFO:teuthology.orchestra.run.smithi198.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T18:45:24.960 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION_CODENAME=focal 2022-02-06T18:45:24.960 INFO:teuthology.orchestra.run.smithi198.stdout:UBUNTU_CODENAME=focal 2022-02-06T18:45:24.960 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server 2022-02-06T18:45:25.069 INFO:teuthology.dispatcher.supervisor:Running job 6665227 2022-02-06T18:45:25.070 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665227 --name benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects} -- /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665227/orig.config.yaml 2022-02-06T18:45:25.073 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2022-02-06_17:57:58-rados-safe_shutdown_v2-distro-basic-smithi/6665227 2022-02-06T18:45:25.074 INFO:teuthology.dispatcher.supervisor:Job PID: 29398 2022-02-06T18:45:25.075 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-07T01:15:35.681 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-07T01:15:35.856 INFO:teuthology.kill:Killing Pids: {29398} 2022-02-07T01:15:37.053 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-07T01:15:37.054 INFO:teuthology.task.internal:roles: ubuntu@smithi198.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-07T01:15:37.054 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:15:37.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2022-02-07T01:15:37.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi198.front.sepia.ceph.com' 2022-02-07T01:15:37.469 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:15:38.361 INFO:teuthology.misc:Compressing logs... 2022-02-07T01:15:48.446 INFO:teuthology.kill:No teuthology processes running 2022-02-07T01:15:49.602 INFO:teuthology.kill:Nuking machines: ['smithi178', 'smithi198'] 2022-02-07T01:15:50.568 INFO:teuthology.kill:2022-02-07 01:15:50,567.567 INFO:teuthology.nuke:targets: 2022-02-07T01:15:50.568 INFO:teuthology.kill: smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbhabG3EgVAeL3yA3oEJOmJcqLPaxFyEBBxP52r1PQst5MAGNYkaRx4sOSdYryrv64PEyQW9jHVC2e+ObtXnMkGVyKkqFap4gldPN3K7RK6oheAUOPStKi+8PuN5zUgH7HQqGoerE3pgl+jm9baBMB+luIIJh7e1dDhovADyL2V8PmPgyweiF63fp3cFapV2mF4SV3bs6Alc/GxL3s8CILs/eyRbaiQqyPLV30FbrEXm4iKjLg5d9lmovMr2ZhnCBu9aRBXxirpNNzhLaj3mzdlOABNips6QRNXOTmNcu4SGeuLPzvV1NXWl665kBSvcq/uhQ/48tFepwuvWKjG0ESQRaX/xANpGLNaX98BlA2jGPMcVUgG6m71ImDEhu/lXxmc385j7KF4QZNcHdQN+PM8Lxd7MT/5dl/kLX5lfRi58byoRUxAnn2tiahpIekcroU+4HmSQFrFT7kpP2ZhE7zB7Rsmc4c+UgbuXqd81pV8eWOyWijUIwf1jLnQA7gVc= 2022-02-07T01:15:50.569 INFO:teuthology.kill: smithi198.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDo/bd+7o23hn6Y9QLGsQDSYghv5paGb4t2tVAlpSE57EpUnZ1HqVii94fgufPYeOIEzfqcSzYg7aknQUFtktl3nOvUzIdfmn8KtYCPp9BGiZF/+bSC44Qrd6sLSgtcDYNF99/q3GN9JlL9C9mb7qCdhbzxtD0Xqad4JJ5BgSBANKF6qmX7bDV/gG4sG8ioOg6qbFWDPO/se1koCBIftFz6VuexXmpseg/r6uyRR/CJaTWhyI4bNc2dOnZQ1XBQgC22n0oHhtgqvVEkLV5sjA09IJb7zPgu9fIB7Tw9hmoJVTfXkTTXRm9accEP7XWQJ9pByH93wQnJtpJQ1kfK9mdacCDxYISHsIrmyU0ZHFb+e7N1ipHoF9v+lj6RThMI8OPUHOkuajj18kcpWjnWqvnU5bpDR5PVb6y9PImYIV2S4cLio3fwQicDuj+rlEAHgmFJZ/zl+P23uOcbuzeX40iXQJLM7pQwHNcC/g4iSAeKTJj9cEJ2R7+aI1IO6zj+mqU= 2022-02-07T01:15:50.595 INFO:teuthology.kill:2022-02-07 01:15:50,594.594 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T01:15:50.599 INFO:teuthology.kill:2022-02-07 01:15:50,599.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-07T01:15:50.677 INFO:teuthology.kill:2022-02-07 01:15:50,677.677 INFO:teuthology.orchestra.console:Power off smithi178 2022-02-07T01:15:50.691 INFO:teuthology.kill:2022-02-07 01:15:50,690.690 INFO:teuthology.orchestra.console:Power off smithi198 2022-02-07T01:15:59.046 INFO:teuthology.kill:2022-02-07 01:15:59,046.046 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2022-02-07T01:15:59.053 INFO:teuthology.kill:2022-02-07 01:15:59,052.052 INFO:teuthology.orchestra.console:Power off for smithi198 completed 2022-02-07T01:15:59.193 INFO:teuthology.kill:2022-02-07 01:15:59,192.192 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com 2022-02-07T01:15:59.197 INFO:teuthology.kill:2022-02-07 01:15:59,197.197 INFO:teuthology.lock.ops:unlocked: smithi198.front.sepia.ceph.com 2022-02-07T01:17:59.545 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-07T01:17:59.654 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi178 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-06_22:53:48-rbd-wip-rbd-quincy-batch-2-distro-default-smithi/6665522 2022-02-07T01:17:59.654 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi198 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-06_22:53:48-rbd-wip-rbd-quincy-batch-2-distro-default-smithi/6665522