2022-04-20T17:20:23.706 INFO:root:teuthology version: 1.1.0-b37d6865 2022-04-20T17:20:23.838 INFO:teuthology.lock.ops:Start node 'smithi019.front.sepia.ceph.com' reimaging 2022-04-20T17:20:23.838 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: reset os type and version on server 2022-04-20T17:20:23.839 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2022-04-20T17:20:23.858 INFO:teuthology.lock.ops:Node 'smithi019.front.sepia.ceph.com' reimaging is complete 2022-04-20T17:20:23.858 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2022-04-20T17:20:23.858 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2022-04-20T17:20:23.858 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-04-20T17:20:23.878 INFO:teuthology.lock.ops:Node 'smithi089.front.sepia.ceph.com' reimaging is complete 2022-04-20T17:20:24.075 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of rhel 8.4 2022-04-20T17:20:24.095 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of rhel 8.4 2022-04-20T17:20:24.428 INFO:teuthology.orchestra.console:Power off smithi019 2022-04-20T17:20:24.445 INFO:teuthology.orchestra.console:Power off smithi089 2022-04-20T17:20:32.821 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2022-04-20T17:20:32.922 INFO:teuthology.orchestra.console:Power on smithi019 2022-04-20T17:20:36.939 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-04-20T17:20:37.040 INFO:teuthology.orchestra.console:Power on smithi089 2022-04-20T17:20:41.296 INFO:teuthology.orchestra.console:Power on for smithi089 completed 2022-04-20T17:20:41.398 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2022-04-20T17:20:41.409 INFO:teuthology.orchestra.console:Power on for smithi019 completed 2022-04-20T17:20:41.510 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2022-04-20T17:24:59.149 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-20T17:24:59.377 INFO:teuthology.provision.fog.smithi019:Node is ready 2022-04-20T17:24:59.393 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2022-04-20T17:24:59.449 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2022-04-20T17:24:59.777 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2022-04-20T17:25:28.822 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-20T17:25:29.059 INFO:teuthology.provision.fog.smithi089:Node is ready 2022-04-20T17:25:29.075 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2022-04-20T17:25:29.132 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2022-04-20T17:25:29.459 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2022-04-20T17:25:29.621 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2022-04-20T17:25:29.622 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-20T17:25:29.644 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2022-04-20T17:25:29.644 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-20T17:25:29.663 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2022-04-20T17:25:29.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2022-04-20T17:25:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2022-04-20T17:25:29.968 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="Red Hat Enterprise Linux" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8.4 (Ootpa)" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:ID="rhel" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="fedora" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8.4" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2022-04-20T17:25:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-04-20T17:25:29.970 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2022-04-20T17:25:29.970 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-04-20T17:25:29.970 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://www.redhat.com/" 2022-04-20T17:25:29.970 INFO:teuthology.orchestra.run.smithi019.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-04-20T17:25:29.970 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-20T17:25:29.971 INFO:teuthology.orchestra.run.smithi019.stdout: 2022-04-20T17:25:29.971 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-20T17:25:29.971 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-04-20T17:25:29.971 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-04-20T17:25:29.971 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-04-20T17:25:29.973 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2022-04-20T17:25:29.994 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2022-04-20T17:25:29.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-04-20T17:25:30.216 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2022-04-20T17:25:30.272 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2022-04-20T17:25:30.272 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.4 (Ootpa)" 2022-04-20T17:25:30.272 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2022-04-20T17:25:30.273 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2022-04-20T17:25:30.273 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.4" 2022-04-20T17:25:30.273 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout: 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-04-20T17:25:30.274 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-04-20T17:25:30.275 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-04-20T17:25:30.275 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-04-20T17:25:30.276 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-04-20T17:25:30.371 INFO:teuthology.dispatcher.supervisor:Running job 6797977 2022-04-20T17:25:30.371 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-20_15:58:35-rados-wip-yuri-testing-2022-04-20-0729-pacific-distro-default-smithi/6797977 --name yuriw-2022-04-20_15:58:35-rados-wip-yuri-testing-2022-04-20-0729-pacific-distro-default-smithi --description rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} -- /home/teuthworker/archive/yuriw-2022-04-20_15:58:35-rados-wip-yuri-testing-2022-04-20-0729-pacific-distro-default-smithi/6797977/orig.config.yaml 2022-04-20T17:25:30.374 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-20_15:58:35-rados-wip-yuri-testing-2022-04-20-0729-pacific-distro-default-smithi/6797977 2022-04-20T17:25:30.374 INFO:teuthology.dispatcher.supervisor:Job PID: 16762 2022-04-20T17:25:30.375 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-04-20T23:55:56.612 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-04-20T23:55:56.808 INFO:teuthology.kill:Killing Pids: {16762} 2022-04-20T23:55:57.896 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-04-20T23:55:57.897 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-04-20T23:55:57.897 INFO:teuthology.misc:Compressing logs... 2022-04-20T23:55:57.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2022-04-20T23:55:58.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-04-20T23:55:58.405 INFO:teuthology.misc:Compressing logs... 2022-04-20T23:55:58.611 INFO:teuthology.misc:Compressing logs... 2022-04-20T23:56:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-04-20T23:56:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-04-20T23:56:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-04-20T23:56:13.809 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-04-20T23:56:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-04-20T23:56:26.441 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-04-20T23:56:26.704 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-04-20T23:56:36.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-04-20T23:56:38.800 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-04-20T23:56:45.604 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-04-20T23:56:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:gzip: /var/log/ceph/714a9894-c0d1-11ec-8c38-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-04-20T23:57:12.786 INFO:teuthology.kill:No teuthology processes running 2022-04-20T23:57:13.782 INFO:teuthology.kill:Nuking machines: ['smithi019', 'smithi089'] 2022-04-20T23:57:14.651 INFO:teuthology.kill:2022-04-20 23:57:14,651.651 INFO:teuthology.nuke:targets: 2022-04-20T23:57:14.651 INFO:teuthology.kill: smithi019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHC4DLnoYUpz42XEiY9qJ/YyVUT9yYpZPO1xTrkMTiJ/bgTGc0zPHsulUSperLdztLM1w9tJNtlyd2AhexFxdqlZsXHglwmV2N2t5TUoiJnPr4b4Eu0rlzZXQdDA4CwDolCedmeS9FkLbTruEf1v7E/ivo2vbfBbxkpLYqtkcN0zAI+rYLBU2zyupZXe0umd+1P4EPLrb8MZED5Md9qZKgPlCC4fPNo1T9GZRM2ua0exnA9X4VZ+wP88A3FlJWe1+wxrYpu75H2nxlsCJxQcGqKViHv4KDKbNnqg4ZTNX4zjxfByN3LDW9GcoLJ3WuAhIoJuN833igM/qmelhc5yEaCMbnmAqPnTge7Rlcd8gzNSjbsBBBrKza6M/u6jvhbMjlc/e9QqJSQplQp0gwYB7MRuHmH8+b9X2GqQnRdLJ6l8vbYAuBGEFdn6jA6AdCr345WyNwJyEUMgqNWEtjLq3+TTEqXDaM4RXMm9lshK178NzucK2AzbPjW/xo9xeVRy8= 2022-04-20T23:57:14.652 INFO:teuthology.kill: smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCn2v9bOmUEzMLZiHYkSqEaRTOuv+lQIXLkyj/fdYx935/K+L8eoNns0zlmhV2b1gjmChPVFWy9nk5lG6fHPZ39/Y5Ov4f16zVHBL3J0NeDg/01nby2O8in/kZf4Ew3hZfs5iJHIdmPFrLvWQqN8hUvAQ9f9QwemnD2tpfLFkc43dj98d5VE8zDUMk/BMLxQV8l4QlnmnF/jbdYX7OTLSkOACy3mxGycvtw3F6LVJJwwGTHaKxJJbOUoS1Eo6qGzmWSKxPt61bBVcyuiV04olWm3mKA0obmQ6oPY3N+GBMldg6KnPxCtNq2FEXYCTj/4gUj67ExIKNwHg7g2ua08KPQmFVe4qnft/X1Mos3MXoscjJ0VFIBkYA8EHLuqQ18vqF9hBqU8lbSc+jbZNUzZpckMtSLqgPQQ3g583vzWxGtzIpxr8SVEdKp2bFjp9VbyUO4vs+TQnSTdqkKnvMwtWzQfHq7uyi6VZFu0nB7/Vx3OSOGatmHJ3HbFA7a+C+40Cc= 2022-04-20T23:57:14.678 INFO:teuthology.kill:2022-04-20 23:57:14,678.678 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-20T23:57:14.681 INFO:teuthology.kill:2022-04-20 23:57:14,681.681 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-20T23:57:14.755 INFO:teuthology.kill:2022-04-20 23:57:14,755.755 INFO:teuthology.orchestra.console:Power off smithi089 2022-04-20T23:57:14.768 INFO:teuthology.kill:2022-04-20 23:57:14,768.768 INFO:teuthology.orchestra.console:Power off smithi019 2022-04-20T23:57:23.131 INFO:teuthology.kill:2022-04-20 23:57:23,131.131 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-04-20T23:57:23.276 INFO:teuthology.kill:2022-04-20 23:57:23,276.276 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com 2022-04-20T23:57:27.246 INFO:teuthology.kill:2022-04-20 23:57:27,245.245 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2022-04-20T23:57:27.390 INFO:teuthology.kill:2022-04-20 23:57:27,390.390 INFO:teuthology.lock.ops:unlocked: smithi019.front.sepia.ceph.com 2022-04-20T23:59:27.691 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15