2022-09-29T05:08:38.481 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-09-29T05:08:38.729 INFO:teuthology.lock.ops:Start node 'smithi083.front.sepia.ceph.com' reimaging 2022-09-29T05:08:38.733 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: reset os type and version on server 2022-09-29T05:08:38.739 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2022-09-29T05:08:38.790 INFO:teuthology.lock.ops:Node 'smithi083.front.sepia.ceph.com' reimaging is complete 2022-09-29T05:08:38.790 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging 2022-09-29T05:08:38.791 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server 2022-09-29T05:08:38.791 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-09-29T05:08:38.831 INFO:teuthology.lock.ops:Node 'smithi153.front.sepia.ceph.com' reimaging is complete 2022-09-29T05:08:39.074 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of centos 8.stream 2022-09-29T05:08:39.124 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream 2022-09-29T05:08:39.488 INFO:teuthology.orchestra.console:Power off smithi083 2022-09-29T05:08:39.576 INFO:teuthology.orchestra.console:Power off smithi153 2022-09-29T05:08:48.012 INFO:teuthology.orchestra.console:Power off for smithi083 completed 2022-09-29T05:08:48.113 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-09-29T05:08:48.115 INFO:teuthology.orchestra.console:Power on smithi083 2022-09-29T05:08:48.215 INFO:teuthology.orchestra.console:Power on smithi153 2022-09-29T05:08:52.546 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2022-09-29T05:08:52.655 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2022-09-29T05:08:56.567 INFO:teuthology.orchestra.console:Power on for smithi083 completed 2022-09-29T05:08:56.668 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish 2022-09-29T05:13:00.867 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-29T05:13:26.227 INFO:teuthology.provision.fog.smithi083:Node is ready 2022-09-29T05:13:26.253 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com 2022-09-29T05:13:26.315 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083 2022-09-29T05:13:26.721 INFO:teuthology.provision.fog.smithi083:Deploy complete! 2022-09-29T05:13:36.996 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-09-29T05:13:37.300 INFO:teuthology.provision.fog.smithi153:Node is ready 2022-09-29T05:13:37.367 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2022-09-29T05:13:37.438 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2022-09-29T05:13:37.826 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2022-09-29T05:13:38.190 INFO:teuthology.lock.ops:Checking smithi083.front.sepia.ceph.com 2022-09-29T05:13:38.191 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-29T05:13:38.217 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2022-09-29T05:13:38.218 INFO:teuthology.lock.ops:New key found. Updating... 2022-09-29T05:13:38.254 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: set os type and version on server 2022-09-29T05:13:38.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2022-09-29T05:13:38.572 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2022-09-29T05:13:38.634 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2022-09-29T05:13:38.635 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="8" 2022-09-29T05:13:38.635 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2022-09-29T05:13:38.636 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2022-09-29T05:13:38.637 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="8" 2022-09-29T05:13:38.637 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el8" 2022-09-29T05:13:38.637 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-29T05:13:38.638 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2022-09-29T05:13:38.644 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-29T05:13:38.644 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2022-09-29T05:13:38.645 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-29T05:13:38.645 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-29T05:13:38.645 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-29T05:13:38.647 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2022-09-29T05:13:38.679 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2022-09-29T05:13:38.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-09-29T05:13:38.950 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2022-09-29T05:13:39.010 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2022-09-29T05:13:39.010 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8" 2022-09-29T05:13:39.011 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2022-09-29T05:13:39.011 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2022-09-29T05:13:39.012 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8" 2022-09-29T05:13:39.012 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2022-09-29T05:13:39.012 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-29T05:13:39.013 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2022-09-29T05:13:39.013 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-29T05:13:39.014 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2022-09-29T05:13:39.015 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-29T05:13:39.015 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-29T05:13:39.015 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-29T05:13:39.016 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-09-29T05:13:39.144 INFO:teuthology.dispatcher.supervisor:Running job 7047304 2022-09-29T05:13:39.145 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047304 --name lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi --description upgrade:telemetry-upgrade/pacific-x/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks} -- /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047304/orig.config.yaml 2022-09-29T05:13:39.154 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-09-29_05:03:04-upgrade:telemetry-upgrade-main-distro-default-smithi/7047304 2022-09-29T05:13:39.155 INFO:teuthology.dispatcher.supervisor:Job PID: 6579 2022-09-29T05:13:39.155 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-09-29T17:14:32.493 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-09-29T17:14:33.126 INFO:teuthology.kill:Killing Pids: {6579} 2022-09-29T17:14:34.806 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2022-09-29T17:14:34.810 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mds.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2022-09-29T17:14:34.819 INFO:teuthology.misc:Compressing logs... 2022-09-29T17:14:34.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2022-09-29T17:14:36.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-09-29T17:14:36.410 INFO:teuthology.misc:Compressing logs... 2022-09-29T17:14:36.901 INFO:teuthology.misc:Compressing logs... 2022-09-29T17:14:52.521 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-09-29T17:14:53.493 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-09-29T17:15:09.431 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-09-29T17:15:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-09-29T17:15:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2022-09-29T17:15:26.962 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-09-29T17:15:30.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-09-29T17:15:41.283 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-09-29T17:15:42.142 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-09-29T17:15:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-09-29T17:15:58.454 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-09-29T17:16:21.901 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/69111522-3fb6-11ed-8431-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-09-29T17:16:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:tar: .: file changed as we read it 2022-09-29T17:16:59.793 INFO:teuthology.kill:No teuthology processes running 2022-09-29T17:17:01.423 INFO:teuthology.kill:Nuking machines: ['smithi083', 'smithi153'] 2022-09-29T17:17:02.718 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_35ea38a9840006713a3d42472a2c536a25e88c15/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-09-29T17:17:02.720 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-09-29T17:17:02.720 INFO:teuthology.kill:2022-09-29 17:17:02,718.718 INFO:teuthology.nuke:targets: 2022-09-29T17:17:02.721 INFO:teuthology.kill: smithi083.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr/JhWmtHMEekWBV5bEXMmOwblnHnXCWHjG06+j5Chpx4UPoXAVHlbwqsRH7KkkeULX7+1p3Jzx0SBz3q3ONlw5vwO7o2wcwG0fTICEx5hqpP4r5NFcH2Muqx57f51Ddj5y8DryGBPbTSy8kEMGChegFc3N7blbqSvKby6nccdPMq+NtY1hlP8i+3xouplad6tcwMoNyndv5r+5M5u0FbP5XyxX1K5UtsQTTIQ137JD5SvwhjdTxIS5qPPoxWY0+bbP8kg49bjP/OOT7+gksHRVzSoYAgbZB4dTR7kkd5SHvSwpB1A3LJxVcd9YIHOy6Jn1o3qnK3kGkfsxGoPs6tw5DlhFPKj6EP2cFsJ+uPLHOun49IDnxbrhZ0UVj2TPhBcEOTeCyZfu9VTfCxbaJXfNIjL7G9OXZImphZfOUc1Q2jG+i+CS3NEtgsYY4ZpHyQkIs8Gkhs4mJSE5a/7tfdCzfPakBx5BHAco9wnIHV4h5eQfzB8XKAKnEO7aSp5pns= 2022-09-29T17:17:02.721 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuCDE7MiWnaxqnxBCgmaVlbjlAJWjWK+frWrSIni4eyA0GtFuaGewEAK8E3cD/4NxiW5Qr9Yjlp9lM6HUy8LUVaeBEkcV6d28Rt4opCrmBPTZRth14miAHFs3yNPYrPpgIPh50EgU+0pomANhRrwlwSrpnT43+TcRBwuQDKje2Xc9QCP2/HyhbV4khamABVGnpv6r3x2UBuZrWKaEVWKzcDsLmavjvWEjgSaGY8PMmId1hJ1VhMjkPQ7Kc6HXDrNLcg88r2gONCa51rvE6PEm/lWioALqa0Is4567gz4jWAQcoK6QrRghBDDC2CEP0+KVBrI8fkXhKzqPLJEBDme9zGOdsPhS396DCo0YjRtwDvJgRedtdiB1rA93qCoF219+a2J3NXEN4jxVZLIzRN1eW0WE6Oh/FsSEo/vuhWNSsupeWc8ima2fX4M2HSuZ3i0Z+ct3ynDUtw0TyWmhMC21Z9z17hI4aefklEVM/26iZ9XAzp7OaUvPA4q+EYG2XTIU= 2022-09-29T17:17:02.722 INFO:teuthology.kill:2022-09-29 17:17:02,718.718 INFO:teuthology.nuke:Checking targets against current locks 2022-09-29T17:17:02.787 INFO:teuthology.kill:2022-09-29 17:17:02,786.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-29T17:17:02.805 INFO:teuthology.kill:2022-09-29 17:17:02,805.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-29T17:17:02.878 INFO:teuthology.kill:2022-09-29 17:17:02,878.878 INFO:teuthology.orchestra.console:Power off smithi083 2022-09-29T17:17:02.898 INFO:teuthology.kill:2022-09-29 17:17:02,897.897 INFO:teuthology.orchestra.console:Power off smithi153 2022-09-29T17:17:11.255 INFO:teuthology.kill:2022-09-29 17:17:11,254.254 INFO:teuthology.orchestra.console:Power off for smithi083 completed 2022-09-29T17:17:11.410 INFO:teuthology.kill:2022-09-29 17:17:11,409.409 INFO:teuthology.lock.ops:unlocked: smithi083.front.sepia.ceph.com 2022-09-29T17:17:15.375 INFO:teuthology.kill:2022-09-29 17:17:15,375.375 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-09-29T17:17:15.531 INFO:teuthology.kill:2022-09-29 17:17:15,531.531 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2022-09-29T17:19:15.922 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-09-29T17:19:16.052 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi083 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-29_16:40:30-rados-wip-all-kickoff-r-distro-default-smithi/7047738 2022-09-29T17:19:16.054 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-09-29_16:40:30-rados-wip-all-kickoff-r-distro-default-smithi/7047738