2021-11-16T14:28:31.881 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-16T14:28:32.011 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging 2021-11-16T14:28:32.013 INFO:teuthology.lock.ops:Updating [ubuntu@smithi173.front.sepia.ceph.com]: reset os type and version on server 2021-11-16T14:28:32.013 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2021-11-16T14:28:32.036 INFO:teuthology.lock.ops:Node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging is complete 2021-11-16T14:28:32.037 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging 2021-11-16T14:28:32.037 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server 2021-11-16T14:28:32.037 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-11-16T14:28:32.058 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete 2021-11-16T14:28:32.236 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 8.3 2021-11-16T14:28:32.272 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 8.3 2021-11-16T14:28:32.564 INFO:teuthology.orchestra.console:Power off smithi173 2021-11-16T14:28:32.590 INFO:teuthology.orchestra.console:Power off smithi174 2021-11-16T14:28:41.019 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2021-11-16T14:28:41.121 INFO:teuthology.orchestra.console:Power on smithi173 2021-11-16T14:28:45.160 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-11-16T14:28:45.261 INFO:teuthology.orchestra.console:Power on smithi174 2021-11-16T14:28:45.375 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2021-11-16T14:28:45.477 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2021-11-16T14:28:53.635 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2021-11-16T14:28:53.737 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2021-11-16T14:32:57.252 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-16T14:33:12.492 INFO:teuthology.provision.fog.smithi173:Node is ready 2021-11-16T14:33:12.508 INFO:teuthology.orchestra.run.smithi173.stdout:smithi133 2021-11-16T14:33:12.564 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2021-11-16T14:33:12.917 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2021-11-16T14:33:43.261 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-16T14:33:43.539 INFO:teuthology.provision.fog.smithi174:Node is ready 2021-11-16T14:33:43.601 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2021-11-16T14:33:43.657 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2021-11-16T14:33:43.979 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2021-11-16T14:33:44.136 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2021-11-16T14:33:44.137 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-16T14:33:44.155 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2021-11-16T14:33:44.155 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-16T14:33:44.174 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2021-11-16T14:33:44.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2021-11-16T14:33:44.440 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2021-11-16T14:33:44.494 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Linux" 2021-11-16T14:33:44.495 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8" 2021-11-16T14:33:44.495 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2021-11-16T14:33:44.495 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2021-11-16T14:33:44.496 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2021-11-16T14:33:44.496 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2021-11-16T14:33:44.496 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-16T14:33:44.496 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2021-11-16T14:33:44.497 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-16T14:33:44.497 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2021-11-16T14:33:44.497 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-16T14:33:44.497 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-16T14:33:44.498 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-16T14:33:44.499 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2021-11-16T14:33:44.524 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2021-11-16T14:33:44.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-11-16T14:33:44.742 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2021-11-16T14:33:44.818 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux" 2021-11-16T14:33:44.818 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8" 2021-11-16T14:33:44.818 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2021-11-16T14:33:44.819 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2021-11-16T14:33:44.819 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8" 2021-11-16T14:33:44.819 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8" 2021-11-16T14:33:44.819 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-16T14:33:44.820 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2021-11-16T14:33:44.820 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-16T14:33:44.820 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2021-11-16T14:33:44.820 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-16T14:33:44.820 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-16T14:33:44.821 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-16T14:33:44.821 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2021-11-16T14:33:44.940 INFO:teuthology.dispatcher.supervisor:Running job 6506606 2021-11-16T14:33:44.940 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_mbenjamin@shatner --archive /home/teuthworker/archive/mbenjamin-2021-11-16_09:04:10-rgw-wip-rgwadmin-logtest-distro-basic-smithi/6506606 --name mbenjamin-2021-11-16_09:04:10-rgw-wip-rgwadmin-logtest-distro-basic-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} -- /home/teuthworker/archive/mbenjamin-2021-11-16_09:04:10-rgw-wip-rgwadmin-logtest-distro-basic-smithi/6506606/orig.config.yaml 2021-11-16T14:33:44.943 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mbenjamin-2021-11-16_09:04:10-rgw-wip-rgwadmin-logtest-distro-basic-smithi/6506606 2021-11-16T14:33:44.944 INFO:teuthology.dispatcher.supervisor:Job PID: 8895 2021-11-16T14:33:44.944 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-17T02:34:39.111 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-17T02:34:39.303 INFO:teuthology.kill:Killing Pids: {8895} 2021-11-17T02:34:40.599 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2021-11-17T02:34:40.600 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-11-17T02:34:40.600 INFO:teuthology.misc:Compressing logs... 2021-11-17T02:34:40.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2021-11-17T02:34:40.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2021-11-17T02:34:41.291 INFO:teuthology.misc:Compressing logs... 2021-11-17T02:34:41.444 INFO:teuthology.misc:Compressing logs... 2021-11-17T02:41:39.251 INFO:teuthology.kill:No teuthology processes running 2021-11-17T02:41:40.406 INFO:teuthology.kill:Nuking machines: ['smithi173', 'smithi174'] 2021-11-17T02:41:41.343 INFO:teuthology.kill:2021-11-17 02:41:41,342.342 INFO:teuthology.nuke:targets: 2021-11-17T02:41:41.344 INFO:teuthology.kill: smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP9u4cPVmg/EHMJ6+X7xDEipYh2ZsFG6jK2Z6bLHRju+5VoWql/ffZV5Jy671Wq2kGTYDl7qVnOFvRhWthAk6mfEyWSPQjoymGvw7qxpK03ZsCgNhvHrotbEOR8H3zK3+3xD8mDMMd++1qWnkbOFofPMgbccrVh93109K88dCYlHVLRtcUa3GIHEMf4Icf5JpoBOvLaHCKQnokA0CmQKnJyrnFPX3LUtrRyqF6wb1pFL/Rs5WGtW21+0NgcBqRqBU9hbDkmr1O5D4WCkwIJn5j5s/WCUj5Q+hLorrawB38ov0XEsC6cEmrGwG8pgZHL4iyX/8Qh9kMkX9S3LlRgVzhk4JFa/bdiveJbg46fUc1cakmzNosNgS56kDfq/UXUQumiKdSAVo+PM5/0TlhoxNdhFmygEpmm/tHRy7vej6g8cRXnlI+DVEwSTUu51ebLGPqVGriDfnJeQcUGVuCcikHL9JyAZBqvc3yN01ttaTrFzq+PJBWgxv3DUbh7mFe3ak= 2021-11-17T02:41:41.344 INFO:teuthology.kill: smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC64FbYpZXhUc0D0oX5/0bmhG01lsYdDmCLUN7Rca9eT5/+q/1NWf/1QYBvETvXcwl/mImvYw7of/wCqyd5JKFheLWjHg9J1CaveS5Fxnpl2nF3HdWnuZCMsxjpvjwXhEZ7gzvFSWlIEFq1GIX3isCVFaQuKOViL8qj21ArKrhV5qsoV7IlYfTxq4qYh2FWnYRRMfJg1jciNqGSHt7nK2AJ47Sv/llLq4fGarE502/Gm1Wg1OAb05J5uf/yTRn3M5928umr8Ha5RewLbRDHcrXwtxbxx8Tql8zNGoD2l24OOm7zPwsPynQhepBbZQWkzkxTNIqYybAF9JjoStjHFUZgSD3SN4ZCDOZhvlWLmXDBwlaEXAmBpnpZLH4AKl2nksPaYsvl4nWNNgXth3d4eIezYUcqQHjkAgZD2txaViRuE/s0JCB7tnFO5W3l6twZbrskNEL+743ZwooQ+VV2nA/h9wsuBbFKq8ofzLu9ihmtkLxY63czHScL6DIbOwv51A8= 2021-11-17T02:41:41.370 INFO:teuthology.kill:2021-11-17 02:41:41,369.369 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-17T02:41:41.371 INFO:teuthology.kill:2021-11-17 02:41:41,371.371 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-17T02:41:41.458 INFO:teuthology.kill:2021-11-17 02:41:41,458.458 INFO:teuthology.orchestra.console:Power off smithi174 2021-11-17T02:41:41.470 INFO:teuthology.kill:2021-11-17 02:41:41,470.470 INFO:teuthology.orchestra.console:Power off smithi173 2021-11-17T02:41:53.933 INFO:teuthology.kill:2021-11-17 02:41:53,932.932 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2021-11-17T02:41:53.938 INFO:teuthology.kill:2021-11-17 02:41:53,938.938 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2021-11-17T02:41:54.077 INFO:teuthology.kill:2021-11-17 02:41:54,077.077 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2021-11-17T02:41:54.125 INFO:teuthology.kill:2021-11-17 02:41:54,125.125 INFO:teuthology.lock.ops:unlocked: smithi173.front.sepia.ceph.com 2021-11-17T02:43:54.509 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-17T02:43:54.604 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi173 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-16_20:03:49-rados-wip-yuri6-testing-2021-11-16-0926-pacific-distro-basic-smithi/6507792 2021-11-17T02:43:54.604 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-16_20:03:49-rados-wip-yuri6-testing-2021-11-16-0926-pacific-distro-basic-smithi/6507792