2022-07-26T19:30:11.304 INFO:root:teuthology version: 1.1.1.dev399+g5f965ce1
2022-07-26T19:30:11.456 INFO:teuthology.lock.ops:Start node 'smithi042.front.sepia.ceph.com' reimaging
2022-07-26T19:30:11.457 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: reset os type and version on server
2022-07-26T19:30:11.457 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2022-07-26T19:30:11.481 INFO:teuthology.lock.ops:Node 'smithi042.front.sepia.ceph.com' reimaging is complete
2022-07-26T19:30:11.481 INFO:teuthology.lock.ops:Start node 'smithi045.front.sepia.ceph.com' reimaging
2022-07-26T19:30:11.482 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: reset os type and version on server
2022-07-26T19:30:11.482 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2022-07-26T19:30:11.505 INFO:teuthology.lock.ops:Node 'smithi045.front.sepia.ceph.com' reimaging is complete
2022-07-26T19:30:11.699 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of centos 8.stream
2022-07-26T19:30:11.718 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of centos 8.stream
2022-07-26T19:30:12.078 INFO:teuthology.orchestra.console:Power off smithi042
2022-07-26T19:30:12.104 INFO:teuthology.orchestra.console:Power off smithi045
2022-07-26T19:30:20.473 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2022-07-26T19:30:20.481 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2022-07-26T19:30:20.575 INFO:teuthology.orchestra.console:Power on smithi042
2022-07-26T19:30:20.583 INFO:teuthology.orchestra.console:Power on smithi045
2022-07-26T19:30:24.857 INFO:teuthology.orchestra.console:Power on for smithi045 completed
2022-07-26T19:30:24.959 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish
2022-07-26T19:30:28.956 INFO:teuthology.orchestra.console:Power on for smithi042 completed
2022-07-26T19:30:29.058 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish
2022-07-26T19:32:41.350 ERROR:teuthology.orchestra.connection:Error authenticating with smithi045.front.sepia.ceph.com: Authentication failed.
2022-07-26T19:34:29.141 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-26T19:34:42.847 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-26T19:34:43.172 INFO:teuthology.provision.fog.smithi042:Node is ready
2022-07-26T19:34:43.193 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com
2022-07-26T19:34:43.254 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042
2022-07-26T19:34:43.601 INFO:teuthology.provision.fog.smithi042:Deploy complete!
2022-07-26T19:34:49.448 INFO:teuthology.provision.fog.smithi045:Node is ready
2022-07-26T19:34:49.471 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com
2022-07-26T19:34:49.530 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045
2022-07-26T19:34:49.901 INFO:teuthology.provision.fog.smithi045:Deploy complete!
2022-07-26T19:34:50.131 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com
2022-07-26T19:34:50.133 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-26T19:34:50.160 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com
2022-07-26T19:34:50.160 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-26T19:34:50.185 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server
2022-07-26T19:34:50.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2022-07-26T19:34:50.365 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64
2022-07-26T19:34:50.426 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream"
2022-07-26T19:34:50.427 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8"
2022-07-26T19:34:50.427 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos"
2022-07-26T19:34:50.427 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora"
2022-07-26T19:34:50.427 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8"
2022-07-26T19:34:50.428 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8"
2022-07-26T19:34:50.428 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 8"
2022-07-26T19:34:50.428 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31"
2022-07-26T19:34:50.429 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-07-26T19:34:50.429 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/"
2022-07-26T19:34:50.429 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-26T19:34:50.429 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-26T19:34:50.429 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-07-26T19:34:50.431 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server
2022-07-26T19:34:50.456 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server
2022-07-26T19:34:50.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2022-07-26T19:34:50.673 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64
2022-07-26T19:34:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream"
2022-07-26T19:34:50.736 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8"
2022-07-26T19:34:50.736 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos"
2022-07-26T19:34:50.736 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora"
2022-07-26T19:34:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8"
2022-07-26T19:34:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8"
2022-07-26T19:34:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8"
2022-07-26T19:34:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31"
2022-07-26T19:34:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-07-26T19:34:50.738 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/"
2022-07-26T19:34:50.738 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-26T19:34:50.738 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-26T19:34:50.738 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-07-26T19:34:50.739 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server
2022-07-26T19:34:50.859 INFO:teuthology.dispatcher.supervisor:Running job 6948451
2022-07-26T19:34:50.859 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948451 --name gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi --description orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} -- /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948451/orig.config.yaml
2022-07-26T19:34:50.864 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2022-07-26_18:31:55-orch:cephadm-wip-guits-testing-2022-07-26-1524-quincy-distro-default-smithi/6948451
2022-07-26T19:34:50.865 INFO:teuthology.dispatcher.supervisor:Job PID: 456
2022-07-26T19:34:50.865 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-07-27T07:35:13.952 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-07-27T07:35:14.130 INFO:teuthology.kill:Killing Pids: {456}
2022-07-27T07:35:15.619 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2022-07-27T07:35:15.620 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2022-07-27T07:35:15.620 INFO:teuthology.misc:Compressing logs...
2022-07-27T07:35:15.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com'
2022-07-27T07:35:15.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com'
2022-07-27T07:35:15.920 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-07-27T07:35:16.043 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-07-27T07:35:16.094 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-07-27T07:35:16.094 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now
2022-07-27T07:35:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-07-27T07:35:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now
2022-07-27T07:35:16.136 INFO:teuthology.misc:Compressing logs...
2022-07-27T07:35:16.481 INFO:teuthology.misc:Compressing logs...
2022-07-27T07:35:18.508 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-mgr.y.log: file size changed while zipping
2022-07-27T07:35:24.914 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-07-27T07:35:25.175 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-mon.c.log: file size changed while zipping
2022-07-27T07:35:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-07-27T07:35:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-07-27T07:35:38.796 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-07-27T07:35:41.634 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-07-27T07:35:50.606 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-mon.a.log: file size changed while zipping
2022-07-27T07:35:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-07-27T07:35:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-mon.b.log: file size changed while zipping
2022-07-27T07:36:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-07-27T07:36:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/023e3c9e-0d1b-11ed-8430-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-07-27T07:37:01.637 INFO:teuthology.kill:No teuthology processes running
2022-07-27T07:37:03.016 INFO:teuthology.kill:Nuking machines: ['smithi042', 'smithi045']
2022-07-27T07:37:04.162 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_5f965ce157b8dcdaf2da92a5b5d1a63e787efdd9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-07-27T07:37:04.163 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-07-27T07:37:04.163 INFO:teuthology.kill:2022-07-27 07:37:04,161.161 INFO:teuthology.nuke:targets:
2022-07-27T07:37:04.163 INFO:teuthology.kill:  smithi042.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIUWgssPo93AMiuQ9WRAyMRIc5Nu0G79VNJsW5sXYntPFg47pA4ww8FdOuylUq94cngZTUa0fo6XXiw/DusLgJF8aaK8OsVeAND3x4dFQMkPSy97SacUyIqIR9Dx0EM7dzyLqkPpnQ1y/K444Mx8sTyzZdKYnyKPgvN3BRVRrh0hMFfZqOn5eNAtnzxYWLVJfsXEHmynLnMIcR5j9KD+OAb+TBqOVPBiBAuZxveg05czyfw9C4mbLd0+pgJl8Is/i7eta80chus2fQnq0s0JGHvdk/5lLbBP2esMHqoygCa/080WBNZXo1t3z8wILyy2GGLYeDDQERUNj0AMsVS+UInOSla/bw6Rea+86Ys9Xbd/vLIBB2g1cWRDIV1f0msGfdv6gLdA1PqBlm4++ZoeJ2h6vIeUm19dl3KpEDUCF6x+tkRH4iM1zdw3lPZiclpYuMqTWBKK1f6xV804nnxvvXikOR5zgVTIG1P6/b84WqduR6yG0AKrcXTV5OHPKKM/k=
2022-07-27T07:37:04.164 INFO:teuthology.kill:  smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrnr3onfAbVDwFiNqzwrE0Wa+wqHnRwzHbSzB6f3JbpsYQj6U3lysvxXIKL/tZlx28MzIBG0SyRude38wL0HoN21vgrgM7WznlDhw+mOizzY89hQAHi1lDMNhkABiPGF3DNqySwa1SJRkS0khdo9az+uTxCj2XEXZoZt6pPOOnD2qF9SsZuFgT88H3Q2qOokJF8h0zfRsIO845QOfFMepMYHiXBcWXr6ULB/k5ERsNBWHdxjQlshcINVlkLkknw6IE7h2Fux8pMrL0eNLhNMLaIc1NOZeIYz5/MuUQYjdmwLqsYyABlsUEmciLxYKxLGTvL0jyGJOkSNZY2brs9h0sExpNdADPUVVUna7IsVtNReadX4z8TJoqH38Z6iYFtHUyhBtKNPsTH+c0hbXATiEj6hdZ1x1MafJoT2Af9noUNUNXFNorBNOELDSibslVpsaW+qDZvykugOVtBEnsS1JaHJSbDfFtPHAj6iUg3sdL6DSDLg0CwCzNT371NEW7RSE=
2022-07-27T07:37:04.164 INFO:teuthology.kill:2022-07-27 07:37:04,162.162 INFO:teuthology.nuke:Checking targets against current locks
2022-07-27T07:37:04.218 INFO:teuthology.kill:2022-07-27 07:37:04,218.218 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-27T07:37:04.249 INFO:teuthology.kill:2022-07-27 07:37:04,249.249 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-27T07:37:04.307 INFO:teuthology.kill:2022-07-27 07:37:04,306.306 INFO:teuthology.orchestra.console:Power off smithi042
2022-07-27T07:37:04.361 INFO:teuthology.kill:2022-07-27 07:37:04,361.361 INFO:teuthology.orchestra.console:Power off smithi045
2022-07-27T07:37:12.738 INFO:teuthology.kill:2022-07-27 07:37:12,737.737 INFO:teuthology.orchestra.console:Power off for smithi045 completed
2022-07-27T07:37:12.897 INFO:teuthology.kill:2022-07-27 07:37:12,895.895 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com
2022-07-27T07:37:16.818 INFO:teuthology.kill:2022-07-27 07:37:16,818.818 INFO:teuthology.orchestra.console:Power off for smithi042 completed
2022-07-27T07:37:16.977 INFO:teuthology.kill:2022-07-27 07:37:16,977.977 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com
2022-07-27T07:39:17.270 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-07-27T07:39:17.373 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi042 but it was locked by another job: /home/teuthworker/archive/bhubbard-2022-07-27_07:20:47-ceph-ansible-wip-octopus-badone-testing-distro-basic-smithi/6949728