2019-10-15T07:51:30.205 INFO:root:teuthology version: 1.0.0-f40fd3f 2019-10-15T07:51:30.206 INFO:teuthology.results:seed: '1521' 2019-10-15T08:17:53.443 INFO:root:teuthology version: 1.0.0-f40fd3f 2019-10-15T08:17:53.447 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2019-10-15T20:17:03.822 INFO:teuthology.results:Tests finished! gathering results... 2019-10-15T20:17:03.881 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/ failed: 3 dead: 6 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [4414506] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:41:30 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414506/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414506/ "2019-10-15T08:42:59.271036+0000 mon.a (mon.0) 201 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [4414502] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:47:17 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414502/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414502/ "2019-10-15T09:03:57.851508+0000 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [4414500] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 07:11:42 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414500/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414500/ Command failed (workunit test rbd/rbd_mirror_fsx_compare.sh) on smithi081 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1531cf5d04fc342b27da186b11f1338e0d19b7eb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\' RBD_MIRROR_USE_RBD_MIRROR=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /ho me/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror _fsx_compare.sh' Dead ================================================================= [4414497] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 11:25:47 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414497/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414497/ psutil.NoSuchProcess process no longer exists (pid=9611) [4414499] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414499/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414499/ [4414505] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 11:41:13 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414505/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414505/ psutil.NoSuchProcess process no longer exists (pid=26211) [4414504] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414504/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414504/ [4414503] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414503/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414503/ [4414501] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414501/ log: http://qa-proxy.ceph.com/teuthology/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414501/ Pass ================================================================= [4414498] rbd/mirror-thrash/{base/install.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_7.yaml} users/mirror.yaml workloads/rbd-mirror-fsx-workunit.yaml} time: 00:46:51 info: http://pulpito.ceph.com/trociny-2019-10-15_07:49:13-rbd-master-distro-basic-smithi/4414498/ 2019-10-15T20:17:04.103 INFO:teuthology.results:starting coverage generation