2022-04-09T16:49:57.565 INFO:root:teuthology version: 1.1.0-a347b2ac
2022-04-09T16:49:57.686 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging
2022-04-09T16:49:57.687 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server
2022-04-09T16:49:57.687 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-04-09T16:49:57.706 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete
2022-04-09T16:49:57.706 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2022-04-09T16:49:57.707 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2022-04-09T16:49:57.707 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-04-09T16:49:57.725 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2022-04-09T16:49:57.924 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 8.stream
2022-04-09T16:49:57.928 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream
2022-04-09T16:49:58.294 INFO:teuthology.orchestra.console:Power off smithi170
2022-04-09T16:49:58.320 INFO:teuthology.orchestra.console:Power off smithi038
2022-04-09T16:50:06.686 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-04-09T16:50:06.693 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-04-09T16:50:06.788 INFO:teuthology.orchestra.console:Power on smithi170
2022-04-09T16:50:06.799 INFO:teuthology.orchestra.console:Power on smithi038
2022-04-09T16:50:15.160 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2022-04-09T16:50:15.167 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2022-04-09T16:50:15.262 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2022-04-09T16:50:15.268 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2022-04-09T16:54:29.579 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-09T16:54:29.860 INFO:teuthology.provision.fog.smithi038:Node is ready
2022-04-09T16:54:29.878 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2022-04-09T16:54:29.938 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2022-04-09T16:54:30.297 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2022-04-09T16:54:44.317 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-09T16:54:44.615 INFO:teuthology.provision.fog.smithi170:Node is ready
2022-04-09T16:54:44.634 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2022-04-09T16:54:44.693 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2022-04-09T16:54:45.049 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2022-04-09T16:54:45.200 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2022-04-09T16:54:45.201 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-09T16:54:45.222 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2022-04-09T16:54:45.222 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-09T16:54:45.240 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2022-04-09T16:54:45.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-04-09T16:54:45.491 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2022-04-09T16:54:45.550 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2022-04-09T16:54:45.550 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8"
2022-04-09T16:54:45.556 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2022-04-09T16:54:45.557 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2022-04-09T16:54:45.557 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8"
2022-04-09T16:54:45.557 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8"
2022-04-09T16:54:45.557 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-09T16:54:45.557 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2022-04-09T16:54:45.558 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-09T16:54:45.558 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2022-04-09T16:54:45.559 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-09T16:54:45.559 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-09T16:54:45.559 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-09T16:54:45.561 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-04-09T16:54:45.578 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2022-04-09T16:54:45.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-04-09T16:54:45.778 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2022-04-09T16:54:45.837 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream"
2022-04-09T16:54:45.837 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2022-04-09T16:54:45.839 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2022-04-09T16:54:45.839 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2022-04-09T16:54:45.839 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2022-04-09T16:54:45.839 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2022-04-09T16:54:45.840 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-09T16:54:45.840 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2022-04-09T16:54:45.840 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-09T16:54:45.840 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2022-04-09T16:54:45.841 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-09T16:54:45.841 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-09T16:54:45.841 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-09T16:54:45.842 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-04-09T16:54:45.933 INFO:teuthology.dispatcher.supervisor:Running job 6784659
2022-04-09T16:54:45.934 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a347b2ac5400a43ba76a678399b1deda7dae1a6c/virtualenv/bin/teuthology -v --owner scheduled_rraja@teuthology --archive /home/teuthworker/archive/rraja-2022-04-09_16:48:13-fs-rraja-wip-53996-distro-default-smithi/6784659 --name rraja-2022-04-09_16:48:13-fs-rraja-wip-53996-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/rraja-2022-04-09_16:48:13-fs-rraja-wip-53996-distro-default-smithi/6784659/orig.config.yaml
2022-04-09T16:54:45.938 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rraja-2022-04-09_16:48:13-fs-rraja-wip-53996-distro-default-smithi/6784659
2022-04-09T16:54:45.939 INFO:teuthology.dispatcher.supervisor:Job PID: 27681
2022-04-09T16:54:45.940 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-09T23:25:14.689 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-09T23:25:14.872 INFO:teuthology.kill:Killing Pids: {27681}
2022-04-09T23:25:16.056 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2']
2022-04-09T23:25:16.057 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5']
2022-04-09T23:25:16.058 INFO:teuthology.misc:Compressing logs...
2022-04-09T23:25:16.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-04-09T23:25:16.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-04-09T23:25:16.571 INFO:teuthology.misc:Compressing logs...
2022-04-09T23:25:16.947 INFO:teuthology.misc:Compressing logs...
2022-04-09T23:25:28.668 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-09T23:25:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-mon.smithi038.log: file size changed while zipping
2022-04-09T23:25:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping
2022-04-09T23:25:38.164 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-09T23:25:41.308 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-mgr.smithi038.ugunvy.log: file size changed while zipping
2022-04-09T23:25:42.121 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-09T23:25:50.522 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-09T23:26:48.133 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-04-09T23:26:58.399 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-09T23:27:31.702 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/c34ad4fa-b826-11ec-8c37-001a4aab830c/ceph-mds.cephfs.smithi038.vlaqjk.log: file size changed while zipping
2022-04-09T23:28:12.666 INFO:teuthology.kill:No teuthology processes running
2022-04-09T23:28:13.757 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi170']
2022-04-09T23:28:14.647 INFO:teuthology.kill:2022-04-09 23:28:14,646.646 INFO:teuthology.nuke:targets:
2022-04-09T23:28:14.648 INFO:teuthology.kill:  smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGrNHFoHohcqpXpDH5YEaB/2P9yryTDOGB+JFUDt6SlI42uOVJtMQJQ1av1mux87nd4CrdjOmXPJiVDJNyoSQCev+aY8gDrhAjnjqPaNGtM0X5n61N9YH/SAI2TCPu20OeWyDMXhDloItMAXm41Ir1YHY3X79Gl7tKp3yGNgseGAc0xbsMctVZgHLK5a3qWe8kPcCJXQrz4SshS7qS1h0A5gAUs1VcXQ9BJOwrf9rJjIraSvYvBGNonpBah2obTawyZ/BTnmiVQ0T9iCfwxh9RGwEuJYlV6T0+iMGnzBPpc4ecCRNoWDyordBJ+KHLZzVG753jUG9JZtUrHlXGwkRyCC5YQ1RNhk2YzV6Mk0GAQucCVwFiQR47G3Fze+sbUw9ZIQ/IBSjqoLL4lsVTm8efYP81GkjG86DcTWrjkrfAF5I/4TBhPdXHOiJZmsy1OnRfsgRQ2C6zZ1tLLtL+OaiYR2KT5wsHGoxFsYfhdXN+xsrnUv1iXsoZd5jWLBJeB6c=
2022-04-09T23:28:14.648 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC32tEdMU5b8+8s4YMNwfQazgQaOL4fhgs0alXi0H3t9ZN3fQfwlHQgYl6UKuHF3Y7h6U4d1A57zZgKWUtiVABVrQo+IZywBtO2EIsHIbXgYwZRKH3ieTPBiK2imxbWX61Gp7UE0Tx7eYXtHhh/wK489pKu+cJQdF7nKwiVxvrd4uZaiLesU2e5NoniYkDxpxImzWDRakyDOdaO3HGl1qY9jGWHgOydI4pEebpX5JrzimdvMtc4tjQcs4rpU9gav9gKBPYsFWevoBPr441oit5JRiX63zxSef/SnMqYU9VC788z/rgJzMdRYUdB/fkLEmXJYsP5sLxqmZ3WV24i2rLyiuVNOWIqdXtZCbHkPm9Z2aKq8w7CGurxu48ATDCk5K5L5U5fgVk8wY6zUkZEI4Ml4pNuo8lOzi9VxywkRv7HMZKhymtqSdzJ38lgLlMUYN4m7l55nFjFV9SbNlWjypTPTr7HoBpvLU3Cl0niBwOZc6bC8okQdg+1hZm+bCWqwY0=
2022-04-09T23:28:14.675 INFO:teuthology.kill:2022-04-09 23:28:14,675.675 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-09T23:28:14.680 INFO:teuthology.kill:2022-04-09 23:28:14,680.680 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-09T23:28:14.759 INFO:teuthology.kill:2022-04-09 23:28:14,758.758 INFO:teuthology.orchestra.console:Power off smithi170
2022-04-09T23:28:14.772 INFO:teuthology.kill:2022-04-09 23:28:14,772.772 INFO:teuthology.orchestra.console:Power off smithi038
2022-04-09T23:28:23.135 INFO:teuthology.kill:2022-04-09 23:28:23,134.134 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-04-09T23:28:23.278 INFO:teuthology.kill:2022-04-09 23:28:23,277.277 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2022-04-09T23:28:27.240 INFO:teuthology.kill:2022-04-09 23:28:27,239.239 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-04-09T23:28:27.383 INFO:teuthology.kill:2022-04-09 23:28:27,383.383 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2022-04-09T23:30:27.646 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-09T23:30:27.762 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi038 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6784361
2022-04-09T23:30:27.762 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-04-09_15:53:21-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6784361