2020-11-14T17:04:39.220 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-14T17:04:39.222 INFO:teuthology.results:seed: '7793' 2020-11-14T18:39:08.659 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-14T18:39:08.693 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-15T06:34:22.818 INFO:teuthology.results:Tests finished! gathering results... 2020-11-15T06:34:33.399 INFO:scrape:Found 64 jobs 2020-11-15T06:34:38.726 INFO:scrape:Found 6 distinct failure reasons 2020-11-15T06:34:38.727 INFO:scrape:Dead: b'2020-11-15T05:18:26.827 DEBUG:teuthology.exit:Finished running handlers' 2020-11-15T06:34:38.727 INFO:scrape:10 jobs: ['5623352', '5623367', '5623360', '5623378', '5623351', '5623398', '5623339', '5623406', '5623410', '5623390'] 2020-11-15T06:34:38.728 INFO:scrape:suites intersection: ['2-partial-upgrade/firsthalf', '3-thrash/default', '5-finish-upgrade', 'start}', 'thrashosds-health}'] 2020-11-15T06:34:38.728 INFO:scrape:suites union: ['1-ceph-install/mimic', '1-luminous-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-ec-workload', '4-workload/{radosbench', '5-finish-upgrade', '6-nautilus', '6.1-msgr2', '7-final-workload', '7-final-workload/{rbd-python', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift-ragweed_check', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'supported-all-distro/rhel_7', 'supported-all-distro/ubuntu_16.04', 'supported-all-distro/ubuntu_latest', 'thrashosds-health}', 'upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack', 'upgrade:mimic-x/stress-split/{0-cluster/{openstack'] 2020-11-15T06:34:38.728 INFO:scrape: 2020-11-15T06:34:38.729 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd_python.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=mimic TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh' 2020-11-15T06:34:38.729 INFO:scrape:2 jobs: ['5623355', '5623342'] 2020-11-15T06:34:38.729 INFO:scrape:suites intersection: ['1-ceph-install/mimic', 'rgw_ragweed_prepare', 'start}'] 2020-11-15T06:34:38.730 INFO:scrape:suites union: ['1-ceph-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '2-workload/{blogbench', '3-thrash/default', '3-upgrade-sequence/upgrade-all', '4-msgr2', '4-nautilus', '4-workload/{radosbench', '5-final-workload/{blogbench', '5-finish-upgrade', '6-nautilus', '6.1-msgr2', '7-final-workload/{rbd-python', 'ec-rados-default', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_cls', 'rbd_import_export', 'readwrite', 'rgw', 'rgw-swift-ragweed_check', 'rgw_ragweed_check', 'rgw_ragweed_prepare', 'rgw_swift}', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest}', 'supported-all-distro/rhel_7', 'test_rbd_api', 'test_rbd_python}', 'thrashosds-health}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack', 'upgrade:mimic-x/stress-split/{0-cluster/{openstack'] 2020-11-15T06:34:38.730 INFO:scrape: 2020-11-15T06:34:38.730 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi067.front.sepia.ceph.com: ['type=AVC msg=audit(1605376201.489:8006): avc: denied { read } for pid=43983 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9105): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14070): avc: denied { read } for pid=140398 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10570): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.819:15756): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8092): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10571): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8088): avc: denied { open } for pid=93033 comm="sadc" path="/proc/93033/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14074): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12610): avc: denied { read } for pid=129624 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8382): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.301:9922): avc: denied { getattr } for pid=109276 comm="sadc" path="/proc/109276/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8102): avc: denied { getattr } for pid=93551 comm="sadc" path="/proc/93551/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8079): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7975): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8093): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8088): avc: denied { read } for pid=93033 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8382): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9105): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8355): avc: denied { getattr } for pid=97081 comm="sadc" path="/proc/97081/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7971): avc: denied { read } for pid=43314 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8354): avc: denied { open } for pid=97081 comm="sadc" path="/proc/97081/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10570): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7975): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14074): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12610): avc: denied { open } for pid=129624 comm="sadc" path="/proc/129624/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8383): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605376201.489:8006): avc: denied { open } for pid=43983 comm="sadc" path="/proc/43983/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.300:9921): avc: denied { open } for pid=109276 comm="sadc" path="/proc/109276/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.820:15757): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605376201.490:8007): avc: denied { getattr } for pid=43983 comm="sadc" path="/proc/43983/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14075): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8079): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9106): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8089): avc: denied { getattr } for pid=93033 comm="sadc" path="/proc/93033/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.819:15756): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8092): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7976): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12611): avc: denied { getattr } for pid=129624 comm="sadc" path="/proc/129624/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7971): avc: denied { open } for pid=43314 comm="sadc" path="/proc/43314/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8101): avc: denied { open } for pid=93551 comm="sadc" path="/proc/93551/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.300:9921): avc: denied { read } for pid=109276 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14070): avc: denied { open } for pid=140398 comm="sadc" path="/proc/140398/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14071): avc: denied { getattr } for pid=140398 comm="sadc" path="/proc/140398/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8080): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8354): avc: denied { read } for pid=97081 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8101): avc: denied { read } for pid=93551 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7972): avc: denied { getattr } for pid=43314 comm="sadc" path="/proc/43314/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1'] 2020-11-15T06:34:38.731 INFO:scrape:1 jobs: ['5623353'] 2020-11-15T06:34:38.731 INFO:scrape:suites: ['1-ceph-install/mimic', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '4-msgr2', '4-nautilus', '5-final-workload/{blogbench', 'ec-rados-default', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check', 'rgw_ragweed_prepare', 'rgw_swift}', 'start}', 'supported-all-distro/rhel_7}', 'test_rbd_api', 'test_rbd_python}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2020-11-15T06:34:38.731 INFO:scrape: 2020-11-15T06:34:38.732 INFO:scrape:Dead: b'2020-11-15T05:28:02.486 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)...' 2020-11-15T06:34:38.732 INFO:scrape:6 jobs: ['5623369', '5623383', '5623343', '5623347', '5623394', '5623348'] 2020-11-15T06:34:38.732 INFO:scrape: 2020-11-15T06:34:38.733 INFO:scrape:Failure: "2020-11-14 18:31:39.964571 mon.c (mon.0) 345 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log 2020-11-15T06:34:38.733 INFO:scrape:2 jobs: ['5623385', '5623411'] 2020-11-15T06:34:38.733 INFO:scrape:suites intersection: ['1-luminous-install/mimic', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-ec-workload', '5-finish-upgrade', '7-final-workload', 'objectstore/bluestore-bitmap', 'start}', 'thrashosds-health}', 'upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack'] 2020-11-15T06:34:38.734 INFO:scrape:suites union: ['1-luminous-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-ec-workload', '5-finish-upgrade', '7-final-workload', 'objectstore/bluestore-bitmap', 'start}', 'supported-all-distro/ubuntu_16.04', 'supported-all-distro/ubuntu_latest', 'thrashosds-health}', 'upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack'] 2020-11-15T06:34:38.734 INFO:scrape: 2020-11-15T06:34:38.734 INFO:scrape:Failure: failed to recover before timeout expired 2020-11-15T06:34:38.735 INFO:scrape:2 jobs: ['5623335', '5623374'] 2020-11-15T06:34:38.735 INFO:scrape:suites intersection: ['1-ceph-install/mimic', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-nautilus', '6.1-msgr2', '7-final-workload/{rbd-python', 'objectstore/bluestore-bitmap', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift-ragweed_check', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'thrashosds-health}', 'upgrade:mimic-x/stress-split/{0-cluster/{openstack'] 2020-11-15T06:34:38.735 INFO:scrape:suites union: ['1-ceph-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-nautilus', '6.1-msgr2', '7-final-workload/{rbd-python', 'objectstore/bluestore-bitmap', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift-ragweed_check', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'thrashosds-health}', 'upgrade:mimic-x/stress-split/{0-cluster/{openstack'] 2020-11-15T06:34:38.735 INFO:scrape: 2020-11-15T06:34:38.741 INFO:teuthology.results:Sending results to ceph-qa@ceph.com: Test Run: teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/ failed: 7 dead: 16 running: 0 waiting: 0 queued: 0 passed: 41 Fail ================================================================= [5623335] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 05:22:05 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623335/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623335/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1afee43934848c7b16546c0ed560c65 failed to recover before timeout expired [5623342] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/centos_latest} ----------------------------------------------------------------- time: 01:58:39 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623342/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623342/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b7a10919e084c6184ef029ec7aad8c7 Command failed (workunit test rbd/test_librbd_python.sh) on smithi023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=mimic TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5623353] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/rhel_7} ----------------------------------------------------------------- time: 03:09:33 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623353/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623353/ SELinux denials found on ubuntu@smithi067.front.sepia.ceph.com: ['type=AVC msg=audit(1605376201.489:8006): avc: denied { read } for pid=43983 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9105): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14070): avc: denied { read } for pid=140398 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10570): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.819:15756): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8092): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10571): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8088): avc: denied { open } for pid=93033 comm="sadc" path="/proc/93033/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14074): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12610): avc: denied { read } for pid=129624 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8382): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.301:9922): avc: denied { getattr } for pid=109276 comm="sadc" path="/proc/109276/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8102): avc: denied { getattr } for pid=93551 comm="sadc" path="/proc/93551/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8079): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7975): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8093): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8088): avc: denied { read } for pid=93033 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8382): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9105): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8355): avc: denied { getattr } for pid=97081 comm="sadc" path="/proc/97081/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7971): avc: denied { read } for pid=43314 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8354): avc: denied { open } for pid=97081 comm="sadc" path="/proc/97081/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605383102.833:10570): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7975): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14074): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12610): avc: denied { open } for pid=129624 comm="sadc" path="/proc/129624/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381002.835:8383): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605376201.489:8006): avc: denied { open } for pid=43983 comm="sadc" path="/proc/43983/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.300:9921): avc: denied { open } for pid=109276 comm="sadc" path="/proc/109276/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.820:15757): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605376201.490:8007): avc: denied { getattr } for pid=43983 comm="sadc" path="/proc/43983/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384602.840:14075): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8079): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382502.827:9106): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378001.617:8089): avc: denied { getattr } for pid=93033 comm="sadc" path="/proc/93033/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605385802.819:15756): avc: denied { read } for pid=30648 comm="ntpd" name="if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378002.840:8092): avc: denied { open } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375602.858:7976): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384001.369:12611): avc: denied { getattr } for pid=129624 comm="sadc" path="/proc/129624/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7971): avc: denied { open } for pid=43314 comm="sadc" path="/proc/43314/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8101): avc: denied { open } for pid=93551 comm="sadc" path="/proc/93551/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605382801.300:9921): avc: denied { read } for pid=109276 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14070): avc: denied { open } for pid=140398 comm="sadc" path="/proc/140398/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605384601.445:14071): avc: denied { getattr } for pid=140398 comm="sadc" path="/proc/140398/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605377402.877:8080): avc: denied { getattr } for pid=30648 comm="ntpd" path="/proc/30648/net/if_inet6" dev="proc" ino=4026532226 scontext=system_u:system_r:ntpd_t:s0 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605381001.191:8354): avc: denied { read } for pid=97081 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605378601.645:8101): avc: denied { read } for pid=93551 comm="sadc" name="dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1605375601.455:7972): avc: denied { getattr } for pid=43314 comm="sadc" path="/proc/43314/net/dev" dev="proc" ino=4026531982 scontext=system_u:system_r:sysstat_t:s0-s0:c0.c1023 tcontext=system_u:object_r:devtty_t:s0 tclass=file permissive=1'] [5623355] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 06:04:07 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623355/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623355/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ed10b22d83849f9ab8703c8191d969a Command failed (workunit test rbd/test_librbd_python.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=mimic TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5623374] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 08:51:06 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623374/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623374/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fe5f7dc26370430783e6e7cb9c3ab7dd failed to recover before timeout expired [5623385] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 00:44:43 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623385/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623385/ "2020-11-14 18:31:39.964571 mon.c (mon.0) 345 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log [5623411] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:44:19 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623411/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623411/ "2020-11-14 20:49:19.329953 mon.c (mon.0) 206 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log Dead ================================================================= [5623339] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623339/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623339/ [5623343] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623343/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623343/ [5623347] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623347/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623347/ [5623348] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623348/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623348/ [5623351] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623351/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623351/ [5623352] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623352/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623352/ [5623360] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623360/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623360/ [5623367] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623367/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623367/ [5623369] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623369/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623369/ [5623378] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623378/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623378/ [5623383] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623383/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623383/ [5623390] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623390/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623390/ [5623394] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623394/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623394/ [5623398] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623398/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623398/ [5623406] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623406/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623406/ [5623410] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-nautilus 6.1-msgr2 7-final-workload/{rbd-python rgw-swift-ragweed_check snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623410/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623410/ Pass ================================================================= [5623334] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/centos_latest} time: 02:49:44 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623334/ [5623336] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 00:49:59 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623336/ [5623337] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/rhel_7} time: 03:10:54 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623337/ [5623338] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04} time: 02:44:15 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623338/ [5623340] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} time: 00:50:44 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623340/ [5623341] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_latest} time: 02:47:56 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623341/ [5623344] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} time: 00:45:14 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623344/ [5623345] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/rhel_7} time: 03:06:19 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623345/ [5623346] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04} time: 02:51:57 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623346/ [5623349] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest} time: 02:50:48 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623349/ [5623350] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/centos_latest} time: 02:57:09 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623350/ [5623354] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04} time: 02:48:25 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623354/ [5623356] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} time: 00:50:44 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623356/ [5623357] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_latest} time: 02:48:58 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623357/ [5623358] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/centos_latest} time: 02:55:26 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623358/ [5623362] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 00:43:39 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623362/ [5623364] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/rhel_7} time: 03:03:32 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623364/ [5623366] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04} time: 02:48:09 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623366/ [5623372] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest} time: 02:44:42 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623372/ [5623373] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/centos_latest} time: 02:52:54 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623373/ [5623375] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 00:48:11 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623375/ [5623376] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/rhel_7} time: 03:12:20 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623376/ [5623377] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04} time: 02:43:38 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623377/ [5623379] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} time: 00:49:02 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623379/ [5623380] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_latest} time: 02:52:45 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623380/ [5623381] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/centos_latest} time: 02:59:25 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623381/ [5623387] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/rhel_7} time: 02:53:26 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623387/ [5623389] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04} time: 02:53:56 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623389/ [5623391] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} time: 00:42:04 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623391/ [5623392] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest} time: 02:42:42 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623392/ [5623393] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/centos_latest} time: 02:50:48 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623393/ [5623395] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} time: 00:47:17 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623395/ [5623396] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/rhel_7} time: 03:13:48 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623396/ [5623397] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04} time: 02:44:36 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623397/ [5623399] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} time: 00:50:49 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623399/ [5623402] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_latest} time: 02:50:09 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623402/ [5623404] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/centos_latest} time: 03:00:07 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623404/ [5623407] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 00:44:05 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623407/ [5623408] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/rhel_7} time: 02:50:25 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623408/ [5623409] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04} time: 02:45:54 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623409/ [5623413] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest} time: 02:46:48 info: http://pulpito.ceph.com/teuthology-2020-11-14_17:03:37-upgrade:mimic-x-nautilus-distro-basic-smithi/5623413/