2021-11-26T01:54:13.295 INFO:root:teuthology version: 1.1.0-a0bc3188 2021-11-26T01:54:13.428 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging 2021-11-26T01:54:13.429 INFO:teuthology.lock.ops:Updating [ubuntu@smithi057.front.sepia.ceph.com]: reset os type and version on server 2021-11-26T01:54:13.430 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-11-26T01:54:13.449 INFO:teuthology.lock.ops:Node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging is complete 2021-11-26T01:54:13.449 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging 2021-11-26T01:54:13.450 INFO:teuthology.lock.ops:Updating [ubuntu@smithi060.front.sepia.ceph.com]: reset os type and version on server 2021-11-26T01:54:13.450 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2021-11-26T01:54:13.470 INFO:teuthology.lock.ops:Node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging is complete 2021-11-26T01:54:13.670 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of ubuntu 20.04 2021-11-26T01:54:13.690 INFO:teuthology.provision.fog.smithi060:Scheduling deploy of ubuntu 20.04 2021-11-26T01:54:14.049 INFO:teuthology.orchestra.console:Power off smithi057 2021-11-26T01:54:14.096 INFO:teuthology.orchestra.console:Power off smithi060 2021-11-26T01:54:22.504 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-11-26T01:54:22.605 INFO:teuthology.orchestra.console:Power on smithi057 2021-11-26T01:54:26.621 INFO:teuthology.orchestra.console:Power off for smithi060 completed 2021-11-26T01:54:26.723 INFO:teuthology.orchestra.console:Power on smithi060 2021-11-26T01:54:30.972 INFO:teuthology.orchestra.console:Power on for smithi057 completed 2021-11-26T01:54:30.980 INFO:teuthology.orchestra.console:Power on for smithi060 completed 2021-11-26T01:54:31.074 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish 2021-11-26T01:54:31.081 INFO:teuthology.provision.fog.smithi060:Waiting for deploy to finish 2021-11-26T01:59:51.468 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-26T01:59:51.568 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-26T01:59:52.053 INFO:teuthology.provision.fog.smithi060:Node is ready 2021-11-26T01:59:52.103 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060.front.sepia.ceph.com 2021-11-26T01:59:52.111 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60 smithi060.front.sepia.ceph.com smithi060 2021-11-26T01:59:52.171 INFO:teuthology.provision.fog.smithi057:Node is ready 2021-11-26T01:59:52.178 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com 2021-11-26T01:59:52.186 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057 2021-11-26T01:59:52.310 INFO:teuthology.provision.fog.smithi060:Deploy complete! 2021-11-26T01:59:52.401 INFO:teuthology.provision.fog.smithi057:Deploy complete! 2021-11-26T01:59:52.608 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com 2021-11-26T01:59:52.609 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-26T01:59:52.630 INFO:teuthology.lock.ops:Checking smithi060.front.sepia.ceph.com 2021-11-26T01:59:52.631 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-26T01:59:52.654 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server 2021-11-26T01:59:52.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2021-11-26T01:59:52.839 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2021-11-26T01:59:52.845 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Ubuntu" 2021-11-26T01:59:52.845 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-26T01:59:52.846 INFO:teuthology.orchestra.run.smithi057.stdout:ID=ubuntu 2021-11-26T01:59:52.846 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE=debian 2021-11-26T01:59:52.846 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-26T01:59:52.846 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="20.04" 2021-11-26T01:59:52.847 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-26T01:59:52.847 INFO:teuthology.orchestra.run.smithi057.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-26T01:59:52.847 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-26T01:59:52.847 INFO:teuthology.orchestra.run.smithi057.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-26T01:59:52.848 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_CODENAME=focal 2021-11-26T01:59:52.848 INFO:teuthology.orchestra.run.smithi057.stdout:UBUNTU_CODENAME=focal 2021-11-26T01:59:52.849 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-11-26T01:59:52.867 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: set os type and version on server 2021-11-26T01:59:52.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2021-11-26T01:59:53.047 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2021-11-26T01:59:53.095 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Ubuntu" 2021-11-26T01:59:53.095 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2021-11-26T01:59:53.095 INFO:teuthology.orchestra.run.smithi060.stdout:ID=ubuntu 2021-11-26T01:59:53.095 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE=debian 2021-11-26T01:59:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2021-11-26T01:59:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="20.04" 2021-11-26T01:59:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.ubuntu.com/" 2021-11-26T01:59:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-11-26T01:59:53.096 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-11-26T01:59:53.097 INFO:teuthology.orchestra.run.smithi060.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-11-26T01:59:53.097 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_CODENAME=focal 2021-11-26T01:59:53.097 INFO:teuthology.orchestra.run.smithi060.stdout:UBUNTU_CODENAME=focal 2021-11-26T01:59:53.098 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2021-11-26T01:59:53.192 INFO:teuthology.dispatcher.supervisor:Running job 6526562 2021-11-26T01:59:53.192 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a0bc318848afe71da1fbf34e1e1b44738d8e3d6d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-25_15:26:23-rgw-pacific-16.2.7_RC1-distro-basic-smithi/6526562 --name yuriw-2021-11-25_15:26:23-rgw-pacific-16.2.7_RC1-distro-basic-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/yuriw-2021-11-25_15:26:23-rgw-pacific-16.2.7_RC1-distro-basic-smithi/6526562/orig.config.yaml 2021-11-26T01:59:53.196 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-25_15:26:23-rgw-pacific-16.2.7_RC1-distro-basic-smithi/6526562 2021-11-26T01:59:53.196 INFO:teuthology.dispatcher.supervisor:Job PID: 32564 2021-11-26T01:59:53.197 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-26T14:00:47.983 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-26T14:00:48.196 INFO:teuthology.kill:Killing Pids: {32564} 2021-11-26T14:00:49.746 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2021-11-26T14:00:49.747 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-11-26T14:00:49.748 INFO:teuthology.misc:Compressing logs... 2021-11-26T14:00:49.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2021-11-26T14:00:49.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2021-11-26T14:00:50.249 INFO:teuthology.misc:Compressing logs... 2021-11-26T14:00:50.389 INFO:teuthology.misc:Compressing logs... 2021-11-26T14:04:44.693 INFO:teuthology.kill:No teuthology processes running 2021-11-26T14:04:46.053 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi060'] 2021-11-26T14:04:47.117 INFO:teuthology.kill:2021-11-26 14:04:47,116.116 INFO:teuthology.nuke:targets: 2021-11-26T14:04:47.118 INFO:teuthology.kill: smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxispNFZJPPVQa7Rxk7t+2QMUsLlxzjZh2rfOfQhCBiBFv4NLbKCuXyWs5LXpZNWyr3yWuV0HDz6VViONbMwPwHwqEp8mLPJwhrfBCswE2+5JbfTN2mjWuBkh42PdGdUs7mlN0onKXeRg8bjTEvFjI7HDMp1Cr4tVoDLZ4I/LPGyupai0UnrXsHYAIzHwiP4Z1a8CyGEV+7dpsh84B8/8X4aCsQw4ll2Oj46Wq47LxBx79L1+9Bz/rM992NKLyVbFl/XMuyG2FxJlqOh6RrFlJtHbC3XYevTqjCAxSdQCaufOP6j4a+X0brUe4m6KuewbTCOJLEYaObqeFv7UqPvRCgzkGaVhyqiop8ZIUZanmb4H01tYU/T9YIHYK31if3XXN/aiKbkfrmN1W8yCV0znHNLRZKdEWEDFamrSHMHtyhL671dR9HOBik77rB3zIM1YgL0N3BKjbNSgumMLVQr2uG/mt5yBaJSV1EKBeCr83Ki3w+hFDcdE8Rqtq5uAbp/0= 2021-11-26T14:04:47.118 INFO:teuthology.kill: smithi060.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBk+/8b2qaUiOgekf+tPZKgft5R0+ucO/b6dOUUF4MlYLBImFjAzBDgpbLC+KU4E9uHP+F5DbNHuh2aWq0x9Dkwo9QY+vib5ktL+6bZ6LM2GtP7kxQeSWmPGTbsi7T+KPi15614lHMMPNP8l4QYWa3Lg3a5WF62tnJ9urDfVlFi2h80kvQTiZY6mY1t7nOxH1XG3EWkpsmEgsxs2UuNns1OY1AoNyTChx6C9bRC6OWRKcD8EIEnPK6jCjh+hBN+yONhFsrscA2hG6Y5hu8UlnZobnJt7tdqGDdm6g77hXx3DHlvlni7fTqUqKt01GXG1ilsDmxGWukb8eXm992LmnxJdR82yddhBcoRG7ZfiipThqSaHsW7uhMk+cavcuafgsekTbKqmktJ2vzDibQ1wfTilQAhKpmcyz66n+nouC9zKWzg8T9h7ZqiWSVqNQjcBp9YWlYnBJONxEpaU0hbqazme2d52PuNPQO9SywyfbUtjShgBYvwizyS5ltMBrNeB8= 2021-11-26T14:04:47.144 INFO:teuthology.kill:2021-11-26 14:04:47,144.144 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-26T14:04:47.147 INFO:teuthology.kill:2021-11-26 14:04:47,147.147 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-26T14:04:47.225 INFO:teuthology.kill:2021-11-26 14:04:47,224.224 INFO:teuthology.orchestra.console:Power off smithi060 2021-11-26T14:04:47.241 INFO:teuthology.kill:2021-11-26 14:04:47,241.241 INFO:teuthology.orchestra.console:Power off smithi057 2021-11-26T14:04:55.607 INFO:teuthology.kill:2021-11-26 14:04:55,607.607 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-11-26T14:04:55.760 INFO:teuthology.kill:2021-11-26 14:04:55,759.759 INFO:teuthology.lock.ops:unlocked: smithi057.front.sepia.ceph.com 2021-11-26T14:04:59.708 INFO:teuthology.kill:2021-11-26 14:04:59,708.708 INFO:teuthology.orchestra.console:Power off for smithi060 completed 2021-11-26T14:04:59.858 INFO:teuthology.kill:2021-11-26 14:04:59,858.858 INFO:teuthology.lock.ops:unlocked: smithi060.front.sepia.ceph.com 2021-11-26T14:07:00.225 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-26T14:07:00.318 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi057 but it was locked by another job: /home/teuthworker/archive/swagner-2021-11-26_13:49:45-orch:cephadm-wip-swagner-testing-2021-11-26-1120-distro-default-smithi/6528119 2021-11-26T14:07:00.318 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi060 but it was locked by another job: /home/teuthworker/archive/swagner-2021-11-26_13:49:45-orch:cephadm-wip-swagner-testing-2021-11-26-1120-distro-default-smithi/6528119