2021-11-08T21:09:52.049 INFO:root:teuthology version: 1.1.0-27954452
2021-11-08T21:09:52.170 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging
2021-11-08T21:09:52.174 INFO:teuthology.lock.ops:Updating [ubuntu@smithi022.front.sepia.ceph.com]: reset os type and version on server
2021-11-08T21:09:52.174 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2021-11-08T21:09:52.195 INFO:teuthology.lock.ops:Node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging is complete
2021-11-08T21:09:52.195 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging
2021-11-08T21:09:52.196 INFO:teuthology.lock.ops:Updating [ubuntu@smithi079.front.sepia.ceph.com]: reset os type and version on server
2021-11-08T21:09:52.196 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2021-11-08T21:09:52.215 INFO:teuthology.lock.ops:Node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging is complete
2021-11-08T21:09:52.424 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 8.3
2021-11-08T21:09:52.427 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 8.3
2021-11-08T21:09:52.780 INFO:teuthology.orchestra.console:Power off smithi079
2021-11-08T21:09:52.822 INFO:teuthology.orchestra.console:Power off smithi022
2021-11-08T21:09:52.931 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2021-11-08T21:09:52.966 INFO:teuthology.orchestra.console:Power off for smithi022 completed
2021-11-08T21:09:53.033 INFO:teuthology.orchestra.console:Power on smithi079
2021-11-08T21:09:53.067 INFO:teuthology.orchestra.console:Power on smithi022
2021-11-08T21:09:57.289 INFO:teuthology.orchestra.console:Power on for smithi079 completed
2021-11-08T21:09:57.390 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish
2021-11-08T21:10:01.433 INFO:teuthology.orchestra.console:Power on for smithi022 completed
2021-11-08T21:10:01.534 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish
2021-11-08T21:12:17.704 ERROR:teuthology.orchestra.connection:Error authenticating with smithi022.front.sepia.ceph.com: Authentication failed.
2021-11-08T21:14:34.175 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-08T21:14:34.489 INFO:teuthology.provision.fog.smithi022:Node is ready
2021-11-08T21:14:34.504 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com
2021-11-08T21:14:34.560 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2021-11-08T21:14:34.893 INFO:teuthology.provision.fog.smithi022:Deploy complete!
2021-11-08T21:15:02.058 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-08T21:15:02.345 INFO:teuthology.provision.fog.smithi079:Node is ready
2021-11-08T21:15:02.400 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com
2021-11-08T21:15:02.455 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079
2021-11-08T21:15:02.773 INFO:teuthology.provision.fog.smithi079:Deploy complete!
2021-11-08T21:15:02.925 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com
2021-11-08T21:15:02.926 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-08T21:15:02.945 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com
2021-11-08T21:15:02.945 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-08T21:15:02.966 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server
2021-11-08T21:15:02.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2021-11-08T21:15:03.245 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64
2021-11-08T21:15:03.300 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Linux"
2021-11-08T21:15:03.300 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="8"
2021-11-08T21:15:03.301 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos"
2021-11-08T21:15:03.301 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora"
2021-11-08T21:15:03.301 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="8"
2021-11-08T21:15:03.301 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el8"
2021-11-08T21:15:03.302 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-08T21:15:03.302 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31"
2021-11-08T21:15:03.302 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-08T21:15:03.303 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/"
2021-11-08T21:15:03.303 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-08T21:15:03.303 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-08T21:15:03.303 INFO:teuthology.orchestra.run.smithi022.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-08T21:15:03.304 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2021-11-08T21:15:03.367 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server
2021-11-08T21:15:03.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2021-11-08T21:15:03.627 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64
2021-11-08T21:15:03.682 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Linux"
2021-11-08T21:15:03.682 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8"
2021-11-08T21:15:03.682 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos"
2021-11-08T21:15:03.682 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora"
2021-11-08T21:15:03.683 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8"
2021-11-08T21:15:03.683 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8"
2021-11-08T21:15:03.683 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Linux 8"
2021-11-08T21:15:03.683 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31"
2021-11-08T21:15:03.684 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-11-08T21:15:03.684 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/"
2021-11-08T21:15:03.684 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-11-08T21:15:03.684 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-11-08T21:15:03.684 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-11-08T21:15:03.685 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2021-11-08T21:15:03.780 INFO:teuthology.dispatcher.supervisor:Running job 6492760
2021-11-08T21:15:03.780 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_27954452159076fda5642b6c0eb0c4998b99d2e4/virtualenv/bin/teuthology -v --owner scheduled_ozeneva@teuthology --archive /home/teuthworker/archive/ozeneva-2021-11-08_20:31:46-rados-wip-omri-tracer-opentelemetry-distro-basic-smithi/6492760 --name ozeneva-2021-11-08_20:31:46-rados-wip-omri-tracer-opentelemetry-distro-basic-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/ozeneva-2021-11-08_20:31:46-rados-wip-omri-tracer-opentelemetry-distro-basic-smithi/6492760/orig.config.yaml
2021-11-08T21:15:03.783 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ozeneva-2021-11-08_20:31:46-rados-wip-omri-tracer-opentelemetry-distro-basic-smithi/6492760
2021-11-08T21:15:03.784 INFO:teuthology.dispatcher.supervisor:Job PID: 13279
2021-11-08T21:15:03.784 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-09T09:15:23.843 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-09T09:15:23.948 INFO:teuthology.kill:Killing Pids: {13279}
2021-11-09T09:15:25.194 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2021-11-09T09:15:25.195 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2021-11-09T09:15:25.195 INFO:teuthology.misc:Compressing logs...
2021-11-09T09:15:25.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2021-11-09T09:15:25.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2021-11-09T09:15:25.705 INFO:teuthology.misc:Compressing logs...
2021-11-09T09:15:26.028 INFO:teuthology.misc:Compressing logs...
2021-11-09T09:15:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mgr.smithi022.glffju.log: file size changed while zipping
2021-11-09T09:15:41.459 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-11-09T09:16:02.306 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mon.smithi022.log: file size changed while zipping
2021-11-09T09:16:16.607 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi079.makjvl.log: file size changed while zipping
2021-11-09T09:16:18.528 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-11-09T09:16:32.803 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-11-09T09:16:35.524 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-11-09T09:16:44.079 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mon.smithi079.log: file size changed while zipping
2021-11-09T09:16:50.180 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-11-09T09:16:52.206 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi022.sgjlhw.log: file size changed while zipping
2021-11-09T09:16:57.757 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-11-09T09:17:02.275 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi079.nweiqb.log: file size changed while zipping
2021-11-09T09:17:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/a4121918-40da-11ec-8c2c-001a4aab830c/ceph-mds.cephfs.smithi022.wmtcwq.log: file size changed while zipping
2021-11-09T09:17:34.789 INFO:teuthology.orchestra.run.smithi022.stderr:tar: .: file changed as we read it
2021-11-09T09:17:53.390 INFO:teuthology.kill:No teuthology processes running
2021-11-09T09:17:54.564 INFO:teuthology.kill:Nuking machines: ['smithi022', 'smithi079']
2021-11-09T09:17:55.562 INFO:teuthology.kill:2021-11-09 09:17:55,562.562 INFO:teuthology.nuke:targets:
2021-11-09T09:17:55.563 INFO:teuthology.kill:  smithi022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDexSHd3tSm69AC9ASr+MkIOxw3kknl/yyA+bq4HynJWl/hCoMNwhlnJ/NUAqXZW5B6nDpipGwqFxgNepXhysTY8Ehb7SKg1coK1qzksVLsDVW1fV7NcOrA0FiDvLS506Itf3WBpw9xrdiDRCNpdByDpVYR32cMJNeDEWFeB0bw+HoBARQOncCst2zU+Akw7oNsOjmtfuWbUrArUzildOdv0Vt0KSIo2HgriY+7q+tzpjYbtNkewW22fj1YLQloz/ZUY98ntjOOuMFQ3FBP1PyvsiQwKW0rsx194VDgrDt/7yxRul/r5+aWzPFscubOSzbkKyOwC15nvmJW8bF7qfejrfbDbznDN2wptmhhw7FidtRupjZwdTYNzat16b023/qGTqpHpRU+o4/Vp095fwG7mKhRJrImYxGpTrNMvin3OLQ6k0IiQen8AQxc02YFt01byk5uJtRlXsL8GpyaTXTgQEi6UlTGBWZjClIFCXEFBQGfNQIwNM8DW5tko/j0uMk=
2021-11-09T09:17:55.564 INFO:teuthology.kill:  smithi079.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUj9K0JG7uASQL7lf5R0ZxJMgNnHWV49iR6HVeJZG0zvjZ0qarf1pXnqpeQOCji73xtHux6I7YbG0L9kXpw0i5MOF6eQ9CRCIgUY9j28GLYzB+mFxLt4z/tan9JSTH6iPNHIWzCG10SPjBfUFJgZDY16iUcPvhmKmcdm2HqtKag6CHArdbCHovSPxUYXV/CPQFoCirsJ+OBWMZE16ojnFYQqqb9/ZoN0WGi+aiy+k7gdUNdBX2ETFJwsCivGo5pxWoSbEjrkxrgWqmp/wjwHejjw3GziHC8Bd2oF4fvKhDo7uzxT4TdyWB/8oC1QWzWwBdg2S8m5XWy6oew+y/2fqjLK9lNikA12lAL1AcW+WxQMDSbkawkgKuuMkt/d/+TGYykjW69EN157QbVklXaYAi5BmARSDdupQzuFoNq00QQYpjTZheQBM+VpijrN6Mf6mXLwpAFku92Akk0pfoJAqd7c7XGKoPtctpQi2WJyaJ3P4/Mcdj3XuWvtQbPApQYIc=
2021-11-09T09:17:55.604 INFO:teuthology.kill:2021-11-09 09:17:55,604.604 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-09T09:17:55.609 INFO:teuthology.kill:2021-11-09 09:17:55,609.609 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-09T09:17:55.696 INFO:teuthology.kill:2021-11-09 09:17:55,695.695 INFO:teuthology.orchestra.console:Power off smithi022
2021-11-09T09:17:55.712 INFO:teuthology.kill:2021-11-09 09:17:55,712.712 INFO:teuthology.orchestra.console:Power off smithi079
2021-11-09T09:18:04.091 INFO:teuthology.kill:2021-11-09 09:18:04,091.091 INFO:teuthology.orchestra.console:Power off for smithi022 completed
2021-11-09T09:18:04.268 INFO:teuthology.kill:2021-11-09 09:18:04,267.267 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com
2021-11-09T09:18:08.208 INFO:teuthology.kill:2021-11-09 09:18:08,207.207 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2021-11-09T09:18:08.353 INFO:teuthology.kill:2021-11-09 09:18:08,353.353 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com
2021-11-09T09:20:08.617 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-09T09:20:08.731 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi022 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-08_20:12:29-rados-wip-yuri4-testing-2021-11-08-1005-pacific-distro-basic-smithi/6491800
2021-11-09T09:20:08.732 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi079 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-08_20:12:29-rados-wip-yuri4-testing-2021-11-08-1005-pacific-distro-basic-smithi/6491799