2022-03-24T03:47:02.904 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-24T03:47:03.024 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging 2022-03-24T03:47:03.025 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T03:47:03.026 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-03-24T03:47:03.044 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete 2022-03-24T03:47:03.044 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging 2022-03-24T03:47:03.045 INFO:teuthology.lock.ops:Updating [ubuntu@smithi154.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T03:47:03.045 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2022-03-24T03:47:03.063 INFO:teuthology.lock.ops:Node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging is complete 2022-03-24T03:47:03.255 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of ubuntu 20.04 2022-03-24T03:47:03.258 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of ubuntu 20.04 2022-03-24T03:47:03.598 INFO:teuthology.orchestra.console:Power off smithi154 2022-03-24T03:47:03.645 INFO:teuthology.orchestra.console:Power off smithi029 2022-03-24T03:47:11.981 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2022-03-24T03:47:12.083 INFO:teuthology.orchestra.console:Power on smithi154 2022-03-24T03:47:16.114 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-03-24T03:47:16.216 INFO:teuthology.orchestra.console:Power on smithi029 2022-03-24T03:47:20.445 INFO:teuthology.orchestra.console:Power on for smithi154 completed 2022-03-24T03:47:20.466 INFO:teuthology.orchestra.console:Power on for smithi029 completed 2022-03-24T03:47:20.547 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2022-03-24T03:47:20.567 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2022-03-24T03:52:27.297 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T03:52:27.756 INFO:teuthology.provision.fog.smithi154:Node is ready 2022-03-24T03:52:27.805 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2022-03-24T03:52:27.812 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2022-03-24T03:52:28.003 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2022-03-24T03:52:56.065 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T03:52:56.631 INFO:teuthology.provision.fog.smithi029:Node is ready 2022-03-24T03:52:56.638 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2022-03-24T03:52:56.689 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2022-03-24T03:52:56.925 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2022-03-24T03:52:57.143 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2022-03-24T03:52:57.144 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T03:52:57.163 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2022-03-24T03:52:57.164 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T03:52:57.181 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2022-03-24T03:52:57.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-03-24T03:52:57.366 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2022-03-24T03:52:57.373 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Ubuntu" 2022-03-24T03:52:57.373 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-24T03:52:57.374 INFO:teuthology.orchestra.run.smithi029.stdout:ID=ubuntu 2022-03-24T03:52:57.374 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE=debian 2022-03-24T03:52:57.374 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-24T03:52:57.374 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="20.04" 2022-03-24T03:52:57.375 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-24T03:52:57.375 INFO:teuthology.orchestra.run.smithi029.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-24T03:52:57.375 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-24T03:52:57.375 INFO:teuthology.orchestra.run.smithi029.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-24T03:52:57.375 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_CODENAME=focal 2022-03-24T03:52:57.376 INFO:teuthology.orchestra.run.smithi029.stdout:UBUNTU_CODENAME=focal 2022-03-24T03:52:57.377 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-03-24T03:52:57.395 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2022-03-24T03:52:57.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2022-03-24T03:52:57.662 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2022-03-24T03:52:57.669 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2022-03-24T03:52:57.669 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-24T03:52:57.669 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2022-03-24T03:52:57.669 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2022-03-24T03:52:57.670 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-24T03:52:57.670 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="20.04" 2022-03-24T03:52:57.670 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-24T03:52:57.671 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-24T03:52:57.671 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-24T03:52:57.671 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-24T03:52:57.671 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=focal 2022-03-24T03:52:57.672 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=focal 2022-03-24T03:52:57.672 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2022-03-24T03:52:57.771 INFO:teuthology.dispatcher.supervisor:Running job 6756563 2022-03-24T03:52:57.771 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756563 --name yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} -- /home/teuthworker/archive/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756563/orig.config.yaml 2022-03-24T03:52:57.774 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756563 2022-03-24T03:52:57.775 INFO:teuthology.dispatcher.supervisor:Job PID: 13663 2022-03-24T03:52:57.775 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-24T10:23:26.551 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-24T10:23:26.721 INFO:teuthology.kill:Killing Pids: {13663} 2022-03-24T10:23:27.864 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-03-24T10:23:27.865 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-03-24T10:23:27.866 INFO:teuthology.misc:Compressing logs... 2022-03-24T10:23:27.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-03-24T10:23:28.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2022-03-24T10:23:28.392 INFO:teuthology.misc:Compressing logs... 2022-03-24T10:23:28.523 INFO:teuthology.misc:Compressing logs... 2022-03-24T10:25:30.802 INFO:teuthology.kill:No teuthology processes running 2022-03-24T10:25:31.955 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi154'] 2022-03-24T10:25:32.999 INFO:teuthology.kill:2022-03-24 10:25:32,999.999 INFO:teuthology.nuke:targets: 2022-03-24T10:25:33.000 INFO:teuthology.kill: smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9cwMnaKnjdHK1VRHA7LrlTZfbet9m+Dwn84Uhj5pCrdmArIOoyvxk3sGex/tvyQ/nd24fAI2cstWAQeWDyZBaWH8Wq3rcXW0GnVHrTXQ/pOWH7EtJOM/dbLowSIv8s6CDqtiBt0+wwizpugee5POVwFt0PstQrj2n5Zk4MvN4CCiX6Xl8CyMBTD963Q4bGIbrgO4cUyjhV5yr0A6qYCe1J1bjSJl0AWtJYlyyEaz6D+6M5FYJ4rxJXFYKgfoWINey0Rd+Joi+Cnm60J2nobCZmNPPKSXMMZZc4M+TXL4K8u7N8PDZuaLfmbi4UP34myjH/GAUXJJhIouzRDNkZhJcSyC+WwcIHiYR/u9DTs/rIFo4d0sI9qkmVLS9hAg062Xf+JZAj9ntR5rrhwyuHAMJvC/Iva6VbOKaYPKebHkPrtLI/jYJconQKtcWU0TdM/UBga9172KseB3HzShHgz3H8wX42npYnzps6apDR3dYakQ4Wj1T/PjgiPCkETJTBtk= 2022-03-24T10:25:33.000 INFO:teuthology.kill: smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD5/gGHfgAWLzRV4ZF0sphcjXppCWIpgJuU8afzuyDzbFbypD+rMu1XMYjq/HKQFQdOaWMHGI3QQD+/rzLb0xLBP1WSCbAargR1eBAx21tQvPYMAZ/ll7qRA/Rd3dUV+tjDSEZorta1yVMfdR+HSBCcJm5n8dyu7gpw2Y7MgymmxC2E2rHt10orph2/qWMEC1ZUXTMlk6d9kTO/QGwtw7Q08v8e6BXn8k9K0JYT1/sLOeZSMBQ/MVJ7liK5KPVPt1oaAsDY1IOyML4JRWkPa5x370k+lSZZ99URro7NQywR0TTPrgF9reEa8HRM+n6a5EiDTJPZWB1CzS0Gbq8cWxi6/vVTwcYAADGAL4z5YaCXmOnfQZKY9HluTXGL/0ml+uYBFaQuVfILTYNGp9vmc9tndejniHowBtyEdGzfIyV6NqsQ9Oe+CaOfjtzsWKJqJatcmmrq+1dpujgk36rRext0E7ZpKk8B1v3+CSlpqAhOszjVG0Rtp8Vko+1wDDT5Hp0= 2022-03-24T10:25:33.025 INFO:teuthology.kill:2022-03-24 10:25:33,025.025 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T10:25:33.028 INFO:teuthology.kill:2022-03-24 10:25:33,028.028 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T10:25:33.103 INFO:teuthology.kill:2022-03-24 10:25:33,103.103 INFO:teuthology.orchestra.console:Power off smithi154 2022-03-24T10:25:33.116 INFO:teuthology.kill:2022-03-24 10:25:33,116.116 INFO:teuthology.orchestra.console:Power off smithi029 2022-03-24T10:25:41.474 INFO:teuthology.kill:2022-03-24 10:25:41,473.473 INFO:teuthology.orchestra.console:Power off for smithi154 completed 2022-03-24T10:25:41.622 INFO:teuthology.kill:2022-03-24 10:25:41,621.621 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com 2022-03-24T10:25:45.588 INFO:teuthology.kill:2022-03-24 10:25:45,587.587 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-03-24T10:25:45.735 INFO:teuthology.kill:2022-03-24 10:25:45,735.735 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2022-03-24T10:27:46.083 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-24T10:27:46.173 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-23_14:05:17-fs-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6755828 2022-03-24T10:27:46.174 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi154 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-23_14:05:17-fs-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6755828