2022-10-25T17:56:46.012 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-25T17:56:46.013 INFO:teuthology.results:seed: '5765' 2022-10-25T18:45:48.373 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-25T18:45:48.398 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-10-25T19:17:50.819 INFO:teuthology.results:Tests finished! gathering results... 2022-10-25T19:17:51.524 INFO:scrape:Found 24 jobs 2022-10-25T19:17:52.196 INFO:scrape:Found 3 distinct failure reasons 2022-10-25T19:17:52.197 INFO:scrape:Failure: Command failed on smithi055 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' 2022-10-25T19:17:52.197 INFO:scrape:12 jobs: ['7080810', '7080792', '7080802', '7080796', '7080798', '7080814', '7080808', '7080812', '7080804', '7080800', '7080794', '7080806'] 2022-10-25T19:17:52.198 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}'] 2022-10-25T19:17:52.198 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}'] 2022-10-25T19:17:52.199 INFO:scrape: 2022-10-25T19:17:52.199 INFO:scrape:Failure: 'wait for operator' reached maximum tries (90) after waiting for 900 seconds 2022-10-25T19:17:52.200 INFO:scrape:6 jobs: ['7080799', '7080803', '7080801', '7080811', '7080815', '7080813'] 2022-10-25T19:17:52.200 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'cluster/1-node', 'k8s/1.21', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-25T19:17:52.201 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-25T19:17:52.202 INFO:scrape: 2022-10-25T19:17:52.202 INFO:scrape:Failure: name 'StringIO' is not defined 2022-10-25T19:17:52.203 INFO:scrape:6 jobs: ['7080809', '7080805', '7080807', '7080793', '7080797', '7080795'] 2022-10-25T19:17:52.203 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'cluster/3-node', 'k8s/1.21', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-25T19:17:52.204 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch:rook:smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-25T19:17:52.205 INFO:scrape: 2022-10-25T19:17:52.206 INFO:teuthology.results:Sending results to choffman@redhat.com: Test Run: choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/ failed: 24 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [7080792] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:20 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080792/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080792/ sentry: https://sentry.ceph.com/organizations/ceph/?query=654142fdb5e84663b3f34b3e91a60ee9 Command failed on smithi189 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080793] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:16:24 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080793/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080793/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8afe130db0b4818bba8605f0adccefd name 'StringIO' is not defined [7080794] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:22 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080794/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080794/ sentry: https://sentry.ceph.com/organizations/ceph/?query=63e62807e57246e4bc7ca5713c06b012 Command failed on smithi164 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080795] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:17:14 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080795/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080795/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c2ab864ee6024feb96ad63bf03f2c0f2 name 'StringIO' is not defined [7080796] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:18 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080796/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080796/ sentry: https://sentry.ceph.com/organizations/ceph/?query=26c55ce017eb4f27baf76bbcf9d95d53 Command failed on smithi062 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080797] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:18:02 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080797/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080797/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7263642e3d44462a3391520402dcc36 name 'StringIO' is not defined [7080798] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:07:01 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080798/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080798/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b76a9319b3154f358f0ae3182e85a2e0 Command failed on smithi055 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080799] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:21:05 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080799/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080799/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e53e54a855c748eca108fdeb94ce6898 'wait for operator' reached maximum tries (90) after waiting for 900 seconds [7080800] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:07:04 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080800/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080800/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9949d3a101c84995ad77ac345ba6bf59 Command failed on smithi102 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080801] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:21:23 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080801/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080801/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b32d0a98812c4f98ba1e290320aab83e 'wait for operator' reached maximum tries (90) after waiting for 900 seconds [7080802] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:56 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080802/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080802/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e9866204626b475cba17d4fba70298ac Command failed on smithi044 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080803] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:21:22 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080803/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080803/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e390f2ef1ec4c0b8daa63c49ef3d756 'wait for operator' reached maximum tries (90) after waiting for 900 seconds [7080804] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:24 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080804/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080804/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8d30922944e84466808fa08ef237d91b Command failed on smithi080 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080805] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:16:26 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080805/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080805/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2adc81b3d87c42f2b4d13654ec0704e8 name 'StringIO' is not defined [7080806] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:34 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080806/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080806/ sentry: https://sentry.ceph.com/organizations/ceph/?query=db9b4bbd756c42c29f0fae3ea078a657 Command failed on smithi027 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080807] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:17:26 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080807/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080807/ sentry: https://sentry.ceph.com/organizations/ceph/?query=33d2bb3d185a45e0874f159ea42c7b35 name 'StringIO' is not defined [7080808] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:11 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080808/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080808/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c908525fa42d426d876fad4870c9e8eb Command failed on smithi107 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080809] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:17:16 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080809/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080809/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a1571ba843545ec8c2c7f142a3a7f9b name 'StringIO' is not defined [7080810] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:07:15 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080810/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080810/ sentry: https://sentry.ceph.com/organizations/ceph/?query=47c1fcc0b457422f8124b6a35d4e3aad Command failed on smithi055 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080811] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:21:06 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080811/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080811/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e21a0f607cd44024b79c88fe7ab8d8c9 'wait for operator' reached maximum tries (90) after waiting for 900 seconds [7080812] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:07:04 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080812/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080812/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29419221f7ab4d9a817cd56fde5c6b64 Command failed on smithi044 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080813] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:21:26 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080813/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080813/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6277b69492f04830839087ada61311fd 'wait for operator' reached maximum tries (90) after waiting for 900 seconds [7080814] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:53 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080814/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080814/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6c0af9c122ef4c8c97c7ce24b6e8e737 Command failed on smithi080 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7080815] orch:rook:smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:21:36 info: http://pulpito.ceph.com/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080815/ log: http://qa-proxy.ceph.com/teuthology/choffman-2022-10-25_17:29:53-orch:rook:smoke-wip-57864-distro-default-smithi/7080815/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ef4e894f755448a3b93b3d3e7fd1e041 'wait for operator' reached maximum tries (90) after waiting for 900 seconds