2020-07-01T03:10:03.383 INFO:root:teuthology version: 1.0.0-645528e 2020-07-01T03:10:03.384 INFO:teuthology.results:subset: '20/9999' 2020-07-01T03:10:03.384 INFO:teuthology.results:seed: '2654' 2020-07-01T03:36:08.425 INFO:root:teuthology version: 1.0.0-645528e 2020-07-01T03:36:08.436 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-07-01T12:26:21.709 INFO:teuthology.results:Tests finished! gathering results... 2020-07-01T12:26:25.191 INFO:scrape:Found 43 jobs 2020-07-01T12:26:48.794 INFO:scrape:Found 5 distinct failure reasons 2020-07-01T12:26:48.796 INFO:scrape:Failure: Command failed on smithi167 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' 2020-07-01T12:26:48.796 INFO:scrape:19 jobs: ['5191777', '5191756', '5191766', '5191764', '5191752', '5191780', '5191769', '5191786', '5191770', '5191750', '5191748', '5191785', '5191776', '5191783', '5191763', '5191761', '5191754', '5191755', '5191760'] 2020-07-01T12:26:48.797 INFO:scrape:suites intersection: ['conf/{client', 'log-config', 'mds', 'mon', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.798 INFO:scrape:suites union: ['clusters/1-mds-1-client', 'clusters/1-mds-2-client', 'conf/{client', 'inline/no', 'inline/yes', 'k-testing}', 'kcephfs/cephfs/{begin', 'kcephfs/mixed-clients/{begin', 'kcephfs/thrash/{begin', 'kclient-overrides/{distro/stock/{k-stock', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'kclient-overrides/{distro/testing/{flavor/ubuntu_latest', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/kclient_workunit_o_trunc}', 'tasks/kclient_workunit_suites_dbench}', 'tasks/kclient_workunit_suites_ffsb}', 'tasks/kclient_workunit_suites_fsstress}', 'tasks/kclient_workunit_suites_fsx}', 'tasks/kclient_workunit_suites_iozone}', 'tasks/kclient_workunit_suites_pjd}', 'tasks/kclient_workunit_trivial_sync}', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'thrash-health-whitelist', 'thrashers/default', 'thrashers/mds', 'thrashers/mon', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/kclient_workunit_suites_ffsb}', 'workloads/kclient_workunit_suites_iozone}'] 2020-07-01T12:26:48.798 INFO:scrape: 2020-07-01T12:26:48.799 INFO:scrape:Failure: "2020-07-01T03:56:44.214724+0000 mon.b (mon.0) 151 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log 2020-07-01T12:26:48.799 INFO:scrape:8 jobs: ['5191758', '5191774', '5191745', '5191778', '5191772', '5191781', '5191779', '5191782'] 2020-07-01T12:26:48.800 INFO:scrape:suites intersection: ['conf/{client', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.800 INFO:scrape:suites union: ['clusters/1-mds-1-client', 'clusters/1-mds-4-client', 'conf/{client', 'inline/no', 'inline/yes', 'k-testing}', 'kcephfs/cephfs/{begin', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/client-recovery}', 'tasks/damage}', 'tasks/kclient_workunit_direct_io}', 'tasks/kclient_workunit_kernel_untar_build}', 'tasks/kclient_workunit_misc}', 'tasks/kclient_workunit_snaps}', 'tasks/kclient_workunit_suites_dbench}', 'tasks/kclient_workunit_suites_fsync}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.800 INFO:scrape: 2020-07-01T12:26:48.801 INFO:scrape:Failure: Scrubbing terminated -- not all pgs were active and clean. 2020-07-01T12:26:48.801 INFO:scrape:1 jobs: ['5191747'] 2020-07-01T12:26:48.802 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'k-testing}', 'kcephfs/thrash/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'thrash-health-whitelist', 'thrashers/default', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/kclient_workunit_suites_ffsb}'] 2020-07-01T12:26:48.802 INFO:scrape: 2020-07-01T12:26:48.803 INFO:scrape:Failure: {'Failure object was': {'smithi183.front.sepia.ceph.com': {'msg': 'Failed to download packages: Cannot download Packages/qemu-kvm-block-rbd-2.12.0-88.module_el8.1.0+297+df420408.3.x86_64.rpm: All mirrors were tried', 'results': [], 'invocation': {'module_args': {'name': ['dnf-utils', 'sysstat', 'libedit', 'boost-thread', 'xfsprogs', 'gdisk', 'parted', 'libgcrypt', 'fuse-libs', 'openssl', 'libuuid', 'podman', 'podman-docker', 'attr', 'ant', 'lsof', 'gettext', 'bc', 'xfsdump', 'blktrace', 'usbredir', 'libev-devel', 'valgrind', 'nfs-utils', 'ncurses-devel', 'gcc', 'git', 'python3-nose', 'python3-virtualenv', 'genisoimage', 'qemu-img', 'qemu-kvm-core', 'qemu-kvm-block-rbd', 'libacl-devel', 'dbench', 'autoconf', 'gdb', 'iozone'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '/')"} 2020-07-01T12:26:48.803 INFO:scrape:1 jobs: ['5191759'] 2020-07-01T12:26:48.803 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/pool-perm}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.804 INFO:scrape: 2020-07-01T12:26:48.804 INFO:scrape:Failure: Command failed on smithi149 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 2020-07-01T12:26:48.805 INFO:scrape:5 jobs: ['5191767', '5191744', '5191751', '5191775', '5191784'] 2020-07-01T12:26:48.805 INFO:scrape:suites intersection: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.806 INFO:scrape:suites union: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/client-limits}', 'tasks/data-scan}', 'tasks/journal-repair}', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-07-01T12:26:48.806 INFO:scrape: 2020-07-01T12:26:48.809 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/ failed: 33 dead: 1 running: 0 waiting: 0 queued: 0 passed: 9 Fail ================================================================= [5191744] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} ----------------------------------------------------------------- time: 00:05:43 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191744/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191744/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=79fd6682d5984d50b9d2f8e979428ca3 Command failed on smithi001 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr- devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5191745] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_dbench} ----------------------------------------------------------------- time: 00:47:09 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191745/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191745/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=049ff552ee854b309f9d56f5731ad9dc "2020-07-01T03:36:26.932739+0000 mon.b (mon.0) 146 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191747] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:44:14 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191747/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191747/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a0fcc2029dbc4d4fa4e9a7914b5e32ac Scrubbing terminated -- not all pgs were active and clean. [5191748] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:18:11 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191748/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191748/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=32ad51ad741d47e6b897fbc59cba33c5 Command failed on smithi154 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191750] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191750/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191750/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9eb53d9b758f42a0a649db3ab6bc2112 Command failed on smithi120 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191751] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} ----------------------------------------------------------------- time: 00:05:48 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191751/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191751/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=dcffb1e4510445bb8daf482e00508833 Command failed on smithi077 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr- devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5191752] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:14:26 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191752/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191752/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=50ac11d536f24d2986f9b207ee5955d4 Command failed on smithi166 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191754] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsx} ----------------------------------------------------------------- time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191754/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191754/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=fab4e8ae95474db09447879cca24a5e5 Command failed on smithi180 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191755] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:14:53 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191755/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191755/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e50fba476dcd479980b612cacc37f9d4 Command failed on smithi191 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191756] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191756/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191756/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9aeaf991ff5249b385b021ea767f34e4 Command failed on smithi016 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191758] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsync} ----------------------------------------------------------------- time: 00:44:01 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191758/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191758/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8dd6ec0ee6144d438c18ee0af0f8fdc0 "2020-07-01T03:56:44.214724+0000 mon.b (mon.0) 151 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191760] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:14:25 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191760/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191760/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0341829badd64d659d29fdd09f2e767b Command failed on smithi189 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191761] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191761/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191761/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=07c0f51cad7142bda1bc3150f36b529b Command failed on smithi078 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191763] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191763/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191763/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=eabf80ee6003495db9827e62bc440122 Command failed on smithi008 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191764] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:18:31 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191764/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191764/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=b772a573c89a4921a520da0eadc0c1ad Command failed on smithi176 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191766] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_trivial_sync} ----------------------------------------------------------------- time: 00:15:03 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191766/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191766/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1381363e909949e890c4879928f48c36 Command failed on smithi040 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191767] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:05:34 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191767/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191767/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f1601efe120e419cae29360c51885a26 Command failed on smithi149 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr- devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5191769] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:16:00 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191769/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191769/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=7ac0328879054734b5e5b70d26edd7b5 Command failed on smithi022 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191770] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:19:10 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191770/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191770/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=107d9bfc9c1f46b79a5269e47d561bf4 Command failed on smithi181 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191772] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_direct_io} ----------------------------------------------------------------- time: 00:47:03 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191772/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191772/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=690257e32d1b42eab0fe63d4b66e744a "2020-07-01T03:45:11.655482+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5191774] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:44:18 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191774/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191774/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ffe2ac3334f84417963bf68ce5538c1c "2020-07-01T03:46:36.998980+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191775] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} ----------------------------------------------------------------- time: 00:05:48 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191775/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191775/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=cec541b49d8841b486cdb98ff6bb6427 Command failed on smithi038 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr- devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5191776] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191776/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191776/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=133aa91d107f4fb6bcd7c07720a6bb64 Command failed on smithi162 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191777] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191777/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191777/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4c096bb4b5a74c5c8742d36e623cb640 Command failed on smithi167 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191778] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_misc} ----------------------------------------------------------------- time: 00:43:28 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191778/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191778/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f2a7f803a4ad47baadee78efef4a8b5e "2020-07-01T03:47:06.176754+0000 mon.b (mon.0) 155 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191779] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:45:10 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191779/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191779/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=06006541983847a59eea346f2e94f190 "2020-07-01T11:50:15.372023+0000 mon.b (mon.0) 149 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5191780] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_o_trunc} ----------------------------------------------------------------- time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191780/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191780/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=663846790d574dc0933a1c67fbc42440 Command failed on smithi104 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191781] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/damage} ----------------------------------------------------------------- time: 00:51:40 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191781/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191781/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1cb1f685a829479ea8824607549d5320 "2020-07-01T10:58:38.220239+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191782] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_snaps} ----------------------------------------------------------------- time: 00:44:13 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191782/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191782/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3be774718c0a43ad885666d1f7c92dc1 "2020-07-01T03:50:31.770583+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5191783] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:18:42 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191783/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191783/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a365d291f8b147bd98ffd4b6b5d73582 Command failed on smithi023 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191784] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} ----------------------------------------------------------------- time: 00:05:59 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191784/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191784/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3f12fd2eec3a4977aa6cfc830ecc6aab Command failed on smithi002 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr- devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5191785] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191785/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191785/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c1e21659eceb49169a74744a35a35a95 Command failed on smithi134 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' [5191786] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_dbench} ----------------------------------------------------------------- time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191786/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191786/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=82f94ef6d941492e9e55c3a3e217eed3 Command failed on smithi140 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout baa1ea6a9656c3db06c66032fa80b476721947ba' Dead ================================================================= [5191759] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} ----------------------------------------------------------------- time: 00:05:01 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191759/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191759/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4b10c6aeb00848c3974452c720741895 {'Failure object was': {'smithi183.front.sepia.ceph.com': {'msg': 'Failed to download packages: Cannot download Packages/qemu-kvm-block- rbd-2.12.0-88.module_el8.1.0+297+df420408.3.x86_64.rpm: All mirrors were tried', 'results': [], 'invocation': {'module_args': {'name': ['dnf-utils', 'sysstat', 'libedit', 'boost-thread', 'xfsprogs', 'gdisk', 'parted', 'libgcrypt', 'fuse-libs', 'openssl', 'libuuid', 'podman', 'podman-docker', 'attr', 'ant', 'lsof', 'gettext', 'bc', 'xfsdump', 'blktrace', 'usbredir', 'libev-devel', 'valgrind', 'nfs-utils', 'ncurses-devel', 'gcc', 'git', 'python3-nose', 'python3-virtualenv', 'genisoimage', 'qemu-img', 'qemu-kvm- core', 'qemu-kvm-block-rbd', 'libacl-devel', 'dbench', 'autoconf', 'gdb', 'iozone'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com _git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te uthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_master/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '/')"} Pass ================================================================= [5191746] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:29:00 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191746/ [5191749] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191749/ [5191753] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191753/ [5191757] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:32:11 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191757/ [5191762] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191762/ [5191765] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:32:20 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191765/ [5191768] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/acls-kernel-client} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191768/ [5191771] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:13:30 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191771/ [5191773] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:18:53 info: http://pulpito.ceph.com/teuthology-2020-06-30_10:25:03-kcephfs-master-testing-basic-smithi/5191773/