2022-03-09T00:35:05.644 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-09T00:35:05.764 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging 2022-03-09T00:35:05.766 INFO:teuthology.lock.ops:Updating [ubuntu@smithi114.front.sepia.ceph.com]: reset os type and version on server 2022-03-09T00:35:05.766 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2022-03-09T00:35:05.786 INFO:teuthology.lock.ops:Node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging is complete 2022-03-09T00:35:05.787 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2022-03-09T00:35:05.788 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2022-03-09T00:35:05.788 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-03-09T00:35:05.807 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2022-03-09T00:35:06.036 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of ubuntu 20.04 2022-03-09T00:35:06.100 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of ubuntu 20.04 2022-03-09T00:35:06.403 INFO:teuthology.orchestra.console:Power off smithi114 2022-03-09T00:35:06.482 INFO:teuthology.orchestra.console:Power off smithi136 2022-03-09T00:35:06.547 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-03-09T00:35:06.648 INFO:teuthology.orchestra.console:Power on smithi114 2022-03-09T00:35:14.850 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-03-09T00:35:14.952 INFO:teuthology.orchestra.console:Power on smithi136 2022-03-09T00:35:15.006 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2022-03-09T00:35:15.107 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2022-03-09T00:35:23.322 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2022-03-09T00:35:23.423 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2022-03-09T00:40:10.450 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-09T00:40:10.910 INFO:teuthology.provision.fog.smithi114:Node is ready 2022-03-09T00:40:10.917 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2022-03-09T00:40:10.965 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2022-03-09T00:40:11.086 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2022-03-09T00:40:17.710 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-09T00:40:18.212 INFO:teuthology.provision.fog.smithi136:Node is ready 2022-03-09T00:40:18.219 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2022-03-09T00:40:18.269 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2022-03-09T00:40:18.459 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2022-03-09T00:40:18.643 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2022-03-09T00:40:18.644 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-09T00:40:18.663 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2022-03-09T00:40:18.663 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-09T00:40:18.682 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2022-03-09T00:40:18.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2022-03-09T00:40:18.874 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2022-03-09T00:40:18.925 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2022-03-09T00:40:18.925 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-09T00:40:18.926 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2022-03-09T00:40:18.926 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2022-03-09T00:40:18.926 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-09T00:40:18.926 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="20.04" 2022-03-09T00:40:18.927 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-09T00:40:18.927 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-09T00:40:18.927 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-09T00:40:18.927 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-09T00:40:18.928 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=focal 2022-03-09T00:40:18.928 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=focal 2022-03-09T00:40:18.929 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2022-03-09T00:40:18.947 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2022-03-09T00:40:18.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-03-09T00:40:19.114 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2022-03-09T00:40:19.120 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2022-03-09T00:40:19.121 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-03-09T00:40:19.121 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2022-03-09T00:40:19.121 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2022-03-09T00:40:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-03-09T00:40:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04" 2022-03-09T00:40:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2022-03-09T00:40:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-03-09T00:40:19.122 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-03-09T00:40:19.123 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-03-09T00:40:19.123 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal 2022-03-09T00:40:19.123 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal 2022-03-09T00:40:19.124 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-03-09T00:40:19.218 INFO:teuthology.dispatcher.supervisor:Running job 6727425 2022-03-09T00:40:19.218 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-08_22:40:01-orch-wip-yuri5-testing-2022-03-07-0958-distro-default-smithi/6727425 --name yuriw-2022-03-08_22:40:01-orch-wip-yuri5-testing-2022-03-07-0958-distro-default-smithi --description orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} -- /home/teuthworker/archive/yuriw-2022-03-08_22:40:01-orch-wip-yuri5-testing-2022-03-07-0958-distro-default-smithi/6727425/orig.config.yaml 2022-03-09T00:40:19.222 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-08_22:40:01-orch-wip-yuri5-testing-2022-03-07-0958-distro-default-smithi/6727425 2022-03-09T00:40:19.222 INFO:teuthology.dispatcher.supervisor:Job PID: 29948 2022-03-09T00:40:19.223 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-09T07:10:48.182 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-09T07:10:48.348 INFO:teuthology.kill:Killing Pids: {29948} 2022-03-09T07:10:49.572 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2022-03-09T07:10:49.573 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2022-03-09T07:10:49.573 INFO:teuthology.misc:Compressing logs... 2022-03-09T07:10:49.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2022-03-09T07:10:49.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-03-09T07:10:50.005 INFO:teuthology.misc:Compressing logs... 2022-03-09T07:10:50.274 INFO:teuthology.misc:Compressing logs... 2022-03-09T07:10:52.256 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-mgr.x.log: file size changed while zipping 2022-03-09T07:11:03.187 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-03-09T07:11:04.588 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-03-09T07:11:07.636 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2022-03-09T07:11:17.126 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-03-09T07:11:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2022-03-09T07:11:22.817 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2022-03-09T07:11:30.164 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-03-09T07:11:31.633 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-03-09T07:11:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-03-09T07:11:41.849 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-03-09T07:11:49.317 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/3ff65cd8-9f43-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-03-09T07:12:17.638 INFO:teuthology.kill:No teuthology processes running 2022-03-09T07:12:18.860 INFO:teuthology.kill:Nuking machines: ['smithi114', 'smithi136'] 2022-03-09T07:12:19.821 INFO:teuthology.kill:2022-03-09 07:12:19,821.821 INFO:teuthology.nuke:targets: 2022-03-09T07:12:19.822 INFO:teuthology.kill: smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGeVwkBhR1INX8ymX3nhXcU0fHJInai4GEH9cECJxo0SQ3EPEqMgKPaw/538e61YpQ5v6KrpbvYJohpz26xdsfvgyDOuZMuZVmMAiqw1jojCqJAbOlGndOr/snPaMiTJnK511h+8f64gyxa1CrNOyJsj55uGxEX1+5XJPyJgHcFJo8Tu9Jdmc4nYv7jQLELwhMKSDitzMLbZn0OdMipDCg+JzjpQJoHA713MDopBEi0dswyEs3muRKUDxGB8VIB08IxCIWHayN3vKApDa8OwHdAJdceJvCsFsZvqp53TsGlbvxEm+Veudqr7X1mdkjbB27xQN8HGEVtyErkOF/o2jsKmLNr+gjc+ZghKinFF5oqcCoOKO/ZGrbk686IIsdHNwvA2FUJF1VZZ7C4loc+3hs1L6ez5K8eya/vF6a3glLz9scwp1/b65g5CnHHaT9i/JBWB0oQGy5O89DB0rlS42ql94x5rwbAGP1VB/lO8XSN0+3sdG0001CiTrDSfHrPT8= 2022-03-09T07:12:19.823 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHRWILLA5Mwpt/vxuo2qdyUXJE1MjfNkP/3iCw3nztDPkh40YgY0SlJ0lpv9xn8F5OK8DgFAHS7G1+zFHZpnWTmKLI3tCu4nv1X8a3ClxelC3y5zn6n3Yl+rtK2Dk4rAr/VsGG2+DbIYpMaf9+tDQIw85+AmC2JmU8ULWwTjVnlQ9gWCQQBT50DOOhPEr4cfAfhl/zkOlzkzx1KDW8s95E6q+CSxpXb57JNYjlirvI8DMDP95k2M27jsvh4KPhJucea94xZcstNGPnLVN7MxiMMEhWRX3ekUpKXlJrNAJRHLEry7OCy5fn03CDNyM9mFc+akPum90Xwq5z0EeHMx4gBzFgInX7nEYnY+JW+8N9O9sjmOPVVHctf11CezPNOIn6jg4/0VIp08fAnaXl4mjgqfVlKmFG9IyskM80+yTbyS1JBEQ6jrmKvuBsuMJebFdN7qXIpnuYuxIMJVI7BdchAZTLvLQfaLU7X5QZc5/Ccn54h1i6E2IvJR28ZI37xJE= 2022-03-09T07:12:19.848 INFO:teuthology.kill:2022-03-09 07:12:19,848.848 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-09T07:12:19.852 INFO:teuthology.kill:2022-03-09 07:12:19,851.851 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-09T07:12:19.928 INFO:teuthology.kill:2022-03-09 07:12:19,928.928 INFO:teuthology.orchestra.console:Power off smithi136 2022-03-09T07:12:19.943 INFO:teuthology.kill:2022-03-09 07:12:19,943.943 INFO:teuthology.orchestra.console:Power off smithi114 2022-03-09T07:12:32.418 INFO:teuthology.kill:2022-03-09 07:12:32,418.418 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-03-09T07:12:32.423 INFO:teuthology.kill:2022-03-09 07:12:32,423.423 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-03-09T07:12:32.568 INFO:teuthology.kill:2022-03-09 07:12:32,568.568 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2022-03-09T07:12:32.572 INFO:teuthology.kill:2022-03-09 07:12:32,572.572 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com 2022-03-09T07:14:32.879 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-09T07:14:32.970 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi114 but it was locked by another job: /home/teuthworker/archive/nmordech-2022-03-09_06:48:27-rados-wip-nitzan-max-pglog-size-distro-default-smithi/6728079 2022-03-09T07:14:32.971 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi136 but it was locked by another job: /home/teuthworker/archive/nmordech-2022-03-09_06:48:27-rados-wip-nitzan-max-pglog-size-distro-default-smithi/6728079