2022-02-03T09:27:45.574 INFO:root:teuthology version: 1.1.0-0f634dc1 2022-02-03T09:27:45.575 INFO:teuthology.results:subset: '26/32' 2022-02-03T09:27:45.576 INFO:teuthology.results:seed: '2567' 2022-02-03T19:18:29.572 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-03T19:18:29.647 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-02-03T20:42:38.358 INFO:teuthology.results:Tests finished! gathering results... 2022-02-03T20:43:29.277 INFO:scrape:Found 436 jobs 2022-02-03T20:47:11.866 INFO:scrape:Found 33 distinct failure reasons 2022-02-03T20:47:11.869 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi005 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2022-02-03T20:47:11.871 INFO:scrape:2 jobs: ['6657887', '6657815'] 2022-02-03T20:47:11.871 INFO:scrape:suites intersection: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.872 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.872 INFO:scrape: 2022-02-03T20:47:11.873 INFO:scrape:Failure: Command failed on smithi041 with status 1: 'sudo yum upgrade -y linux-firmware' 2022-02-03T20:47:11.873 INFO:scrape:17 jobs: ['6658037', '6658044', '6658051', '6658013', '6658048', '6658021', '6658014', '6658030', '6658022', '6658033', '6658031', '6658025', '6658050', '6658018', '6658041', '6658036', '6658016'] 2022-02-03T20:47:11.874 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.875 INFO:scrape:suites union: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/fs/snaps}}', '2-workunit/suites/fsstress}}', '2-workunit/suites/iozone}}', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/thrash/multifs/{begin', 'fs/thrash/workloads/{begin', 'fs/traceless/{begin', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{secure}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'overrides/{no_client_pidfile', 'overrides/{pg-warn', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'roles', 'scrub/yes', 'session_timeout', 'standby-replay', 'syntax', 'tasks/cfuse_workunit_suites_blogbench', 'tasks/pool-perm}', 'tasks/recovery-fs}', 'tasks/{0-check-counter', 'tasks/{0-from/pacific', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'traceless/50pc}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/norstats}', 'workunit/suites/dbench}', 'workunit/suites/fsx}', 'workunit/suites/iozone}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.875 INFO:scrape: 2022-02-03T20:47:11.876 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on smithi104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2022-02-03T20:47:11.876 INFO:scrape:22 jobs: ['6657670', '6658040', '6657651', '6657967', '6657985', '6657813', '6657835', '6657989', '6658003', '6657885', '6657942', '6657908', '6657699', '6657693', '6658015', '6657862', '6657980', '6658055', '6657840', '6657955', '6657789', '6657933'] 2022-02-03T20:47:11.877 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{frag', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.878 INFO:scrape:suites union: ['2-workunit/suites/pjd}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'msgr-failures/none', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.880 INFO:scrape: 2022-02-03T20:47:11.880 INFO:scrape:Failure: {'Failure object was': {'smithi190.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} 2022-02-03T20:47:11.881 INFO:scrape:37 jobs 2022-02-03T20:47:11.882 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.882 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/fs/snaps}}', '2-workunit/fs/trivial_sync}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/fsstress}}', '2-workunit/suites/iozone}}', '2-workunit/suites/pjd}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/thrash/multifs/{begin', 'fs/thrash/workloads/{begin', 'fs/traceless/{begin', 'fs/verify/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'overrides/{mon-debug', 'overrides/{no_client_pidfile', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/cfuse_workunit_suites_blogbench', 'tasks/dbench', 'tasks/quota}', 'tasks/scrub}', 'tasks/strays}', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'traceless/50pc}', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/direct_io}', 'workunit/fs/norstats}', 'workunit/fs/test_o_trunc}', 'workunit/kernel_untar_build}', 'workunit/suites/ffsb}', 'workunit/suites/fsstress}', 'workunit/suites/iogen}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.883 INFO:scrape: 2022-02-03T20:47:11.883 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh' 2022-02-03T20:47:11.884 INFO:scrape:1 jobs: ['6657635'] 2022-02-03T20:47:11.884 INFO:scrape:suites: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.885 INFO:scrape: 2022-02-03T20:47:11.885 INFO:scrape:Crash: "2022-02-03T13:39:22.932742+0000 mon.a (mon.0) 2322 : cluster [WRN] Replacing daemon mds.c as rank 0 with standby daemon mds.a" in cluster log 2022-02-03T20:47:11.886 INFO:scrape:ceph version 17.0.0-10531-g09fc2e9e (09fc2e9e470435f1e6dbf462a51853391f4a4cc0) quincy (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f1b0a8013c0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f1b0aac5834] 5: /usr/lib/ceph/libceph-common.so.2(+0x27b9de) [0x7f1b0aac59de] 6: (MDSRankDispatcher::handle_mds_map(boost::intrusive_ptr const&, MDSMap const&)+0x2df7) [0x7f1b0b80ddc7] 7: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x107b) [0x7f1b0b7e467b] 8: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x7f1b0b7e61ef] 9: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x7f1b0b7e6b86] 10: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f1b0ad5d8d0] 11: (DispatchQueue::entry()+0x58f) [0x7f1b0ad5b16f] 12: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1b0ae29441] 13: /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609) [0x7f1b0a7f5609] 14: clone() 2022-02-03T20:47:11.886 INFO:scrape:10 jobs: ['6657897', '6657949', '6657850', '6657921', '6658019', '6657907', '6657706', '6657920', '6657916', '6658006'] 2022-02-03T20:47:11.887 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.887 INFO:scrape:suites union: ['1-client', '2-upgrade', '2-workunit/cfuse_workunit_snaptests}}', '3-compat_client/pacific}}', 'centos_latest', 'clusters/1-mds-2-client-micro', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/multifs/{begin', 'fs/thrash/multifs/{begin', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'overrides/{mon-debug', 'overrides/{multimds/no', 'overrides/{no_client_pidfile', 'pg-warn', 'rhel_8}', 'session_timeout', 'tasks/client-recovery}', 'tasks/damage}', 'tasks/data-scan}', 'tasks/failover}', 'tasks/fragment}', 'tasks/openfiletable}', 'tasks/{0-octopus', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.888 INFO:scrape: 2022-02-03T20:47:11.890 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi055.front.sepia.ceph.com: ['type=AVC msg=audit(1643891511.738:2670): avc: denied { setgid } for pid=10476 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891478.794:2659): avc: denied { setgid } for pid=9760 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891513.649:2672): avc: denied { setgid } for pid=10485 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891480.614:2664): avc: denied { setgid } for pid=9780 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] 2022-02-03T20:47:11.891 INFO:scrape:85 jobs 2022-02-03T20:47:11.892 INFO:scrape:suites intersection: [] 2022-02-03T20:47:11.893 INFO:scrape:suites union: ['1-client', '1-mount', '1-ranks/1', '1-ranks/2', '1-rename', '1-upgrade}}', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-upgrade', '2-workload/dbench', '2-workunit/fs/snaps}}', '2-workunit/fs/trivial_sync}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/fsstress}}', '2-workunit/suites/iozone}}', '2-workunit/suites/pjd}}', '3-compat_client/no}}', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'centos_latest', 'cephfs-mirror/three-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'clusters/1-mds-1-client-coloc', 'clusters/1-mds-2-client-micro', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'clusters/fixed-2-ucephfs', 'clusters/small-cluster', 'conf/{client', 'distro/single-container-host', 'distro/single-container-host}', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/32bits/{begin', 'fs/bugs/client_trim_caps/{begin', 'fs/cephadm/multivolume/{0-start', 'fs/cephadm/renamevolume/{0-start', 'fs/functional/{begin', 'fs/libcephfs/{begin', 'fs/mirror-ha/{begin', 'fs/mixed-clients/{begin', 'fs/permission/{begin', 'fs/snaps/{begin', 'fs/thrash/workloads/{begin', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/upgrade/nofs/{bluestore-bitmap', 'fs/verify/{begin', 'fs/volumes/{begin', 'fs/workload/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'no-mds-cluster', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/whitelist_health}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{faked-ino', 'overrides/{frag', 'overrides/{mon-debug', 'overrides/{multimds/no', 'overrides/{multimds/yes', 'overrides/{no_client_pidfile', 'overrides/{osd-asserts', 'overrides/{pg-warn', 'overrides/{whitelist_health', 'overrides/{whitelist_health}', 'pg-warn', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'roles', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'supported-random-distro$/{centos_8}', 'syntax', 'tasks/acls}', 'tasks/alternate-pool}', 'tasks/auto-repair}', 'tasks/cfuse_workunit_suites_fsstress}', 'tasks/cfuse_workunit_suites_pjd}', 'tasks/client-readahead}', 'tasks/client}', 'tasks/exports}', 'tasks/fsstress', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'tasks/libcephfs_python}', 'tasks/mds-flush}', 'tasks/mds_creation_retry}', 'tasks/multimds_misc}', 'tasks/trim-i24137}', 'tasks/volumes/{overrides', 'tasks/workunit/snaps}', 'tasks/{0-check-counter', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'tasks/{0-octopus', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'test/snapshot}}', 'thrashosds-health', 'validater/lockdep}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/cephfs-mirror-ha-workunit}', 'workunit/direct_io}', 'workunit/fs/test_o_trunc}', 'workunit/kernel_untar_build}', 'workunit/suites/blogbench}', 'workunit/suites/ffsb}', 'workunit/suites/fsx}', 'workunit/suites/fsync-tester}', 'workunit/suites/iogen}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.894 INFO:scrape: 2022-02-03T20:47:11.894 INFO:scrape:Failure: Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' 2022-02-03T20:47:11.895 INFO:scrape:4 jobs: ['6657882', '6657811', '6657953', '6657668'] 2022-02-03T20:47:11.895 INFO:scrape:suites intersection: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/norstats}'] 2022-02-03T20:47:11.896 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/norstats}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.896 INFO:scrape: 2022-02-03T20:47:11.897 INFO:scrape:Failure: Failure object was: {'smithi137.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:11.323100', 'end': '2022-02-03 11:41:56.435457', 'delta': '0:08:45.112357', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi005.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:18.116903', 'end': '2022-02-03 11:42:03.030919', 'delta': '0:08:44.914016', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') 2022-02-03T20:47:11.897 INFO:scrape:17 jobs: ['6657750', '6657734', '6657757', '6657712', '6657723', '6657763', '6657740', '6657769', '6657744', '6657726', '6657764', '6657751', '6657727', '6657724', '6657743', '6657761', '6657720'] 2022-02-03T20:47:11.898 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.899 INFO:scrape:suites union: ['1-client', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-upgrade', '2-workunit/fs/trivial_sync}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/pjd}}', '3-client-upgrade', '3-upgrade-with-workload', '3-verify}', '4-compat_client', '4-verify}}', '5-client-sanity}}', 'centos_8.stream_container_tools', 'centos_latest', 'clusters/1-mds-2-client-micro', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/permission/{begin', 'fs/thrash/workloads/{begin', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'msgr-failures/none', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'overrides/{multimds/no', 'overrides/{no_client_pidfile', 'overrides/{pg-warn', 'overrides/{whitelist_health', 'pg-warn', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'roles', 'scrub/no', 'session_timeout', 'standby-replay', 'syntax', 'tasks/cfuse_workunit_misc}', 'tasks/sessionmap}', 'tasks/test_journal_migration}', 'tasks/{0-check-counter', 'tasks/{0-from/v16.2.4', 'tasks/{0-octopus', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'workunit/suites/blogbench}', 'workunit/suites/fsstress}', 'workunit/suites/iogen}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.899 INFO:scrape: 2022-02-03T20:47:11.900 INFO:scrape:Failure: Test failure: test_mirroring_with_filesystem_reset (tasks.cephfs.test_admin.TestMirroringCommands) 2022-02-03T20:47:11.900 INFO:scrape:1 jobs: ['6657817'] 2022-02-03T20:47:11.901 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'tasks/admin}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.901 INFO:scrape: 2022-02-03T20:47:11.902 INFO:scrape:Crash: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2022-02-03T20:47:11.907 INFO:scrape:ceph version 17.0.0-10531-g09fc2e9e (09fc2e9e470435f1e6dbf462a51853391f4a4cc0) quincy (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f2d7a5263c0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f2d7a7ea834] 5: /usr/lib/ceph/libceph-common.so.2(+0x27b9de) [0x7f2d7a7ea9de] 6: (CDir::fetch_keys(std::vector > const&, MDSContext*)+0x644) [0x557b0888d304] 7: (MDCache::open_undef_inodes_dirfrags()+0x47b) [0x557b0873695b] 8: (MDCache::rejoin_gather_finish()+0xa8) [0x557b08757508] 9: (MDCache::rejoin_send_rejoins()+0x2832) [0x557b087551d2] 10: (MDCache::process_imported_caps()+0x1262) [0x557b087569a2] 11: (Context::complete(int)+0xd) [0x557b085d344d] 12: (MDSContext::complete(int)+0x61) [0x557b08943cc1] 13: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x90) [0x557b08617e40] 14: (OpenFileTable::_open_ino_finish(inodeno_t, int)+0x119) [0x557b08977679] 15: (OpenFileTable::_prefetch_inodes()+0x2d8) [0x557b08977af8] 16: (Context::complete(int)+0xd) [0x557b085d344d] 17: (MDSContext::complete(int)+0x61) [0x557b08943cc1] 18: (C_GatherBase::sub_finish(MDSContext*, int)+0xfd) [0x557b0861dc4d] 19: (C_GatherBase::C_GatherSub::complete(int)+0x25) [0x557b0861dff5] 20: (MDSRank::_advance_queues()+0x88) [0x557b085e1668] 21: (MDSRank::ProgressThread::entry()+0x29b) [0x557b085e1e9b] 22: /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609) [0x7f2d7a51a609] 23: clone() 2022-02-03T20:47:11.907 INFO:scrape:1 jobs: ['6657672'] 2022-02-03T20:47:11.908 INFO:scrape:suites: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.908 INFO:scrape: 2022-02-03T20:47:11.909 INFO:scrape:Crash: Command failed (workunit test fs/misc/rstats.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' 2022-02-03T20:47:11.909 INFO:scrape:ceph version 17.0.0-10531-g09fc2e9e (09fc2e9e470435f1e6dbf462a51853391f4a4cc0) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f84f46a5b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b8) [0x7f84f56c19f2] 5: (Locker::simple_lock(SimpleLock*, bool*)+0x588) [0x5558161a4108] 6: (Locker::xlock_start(SimpleLock*, boost::intrusive_ptr&)+0x380) [0x5558161a5dd0] 7: (Locker::acquire_locks(boost::intrusive_ptr&, MutationImpl::LockOpVec&, CInode*, bool)+0x24a0) [0x5558161bb8c0] 8: (Server::xlock_nestlock(boost::intrusive_ptr&, CInode*)+0xd2) [0x55581600fab2] 9: (Server::handle_set_vxattr(boost::intrusive_ptr&, CInode*)+0x14d4) [0x555816036364] 10: (Server::handle_client_setxattr(boost::intrusive_ptr&)+0xcd) [0x55581603979d] 11: (Server::dispatch_client_request(boost::intrusive_ptr&)+0xc7b) [0x5558160600ab] 12: (Server::handle_client_request(boost::intrusive_ptr const&)+0x566) [0x55581606bda6] 13: (Server::dispatch(boost::intrusive_ptr const&)+0x133) [0x555816074fe3] 14: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd3c) [0x555815fcb74c] 15: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7cb) [0x555815fce39b] 16: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555815fce9bc] 17: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x108) [0x555815fbd8b8] 18: (DispatchQueue::entry()+0x14fa) [0x7f84f594740a] 19: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f84f59fd891] 20: /lib64/libpthread.so.0(+0x814a) [0x7f84f469b14a] 21: clone() 2022-02-03T20:47:11.910 INFO:scrape:1 jobs: ['6657857'] 2022-02-03T20:47:11.910 INFO:scrape:suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag', 'ranks/3', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'wsync/{no}}'] 2022-02-03T20:47:11.911 INFO:scrape: 2022-02-03T20:47:11.911 INFO:scrape:Failure: No module named 'tasks' 2022-02-03T20:47:11.912 INFO:scrape:6 jobs: ['6657814', '6657790', '6657788', '6657792', '6657793', '6657778'] 2022-02-03T20:47:11.912 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.912 INFO:scrape:suites union: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-workunit/fs/snaps}}', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/snaps/{begin', 'fs/thrash/workloads/{begin', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/volumes/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'overrides/{no_client_pidfile', 'overrides/{pg-warn', 'overrides/{whitelist_health', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/1', 'ranks/5', 'rhel_8}', 'roles', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'syntax', 'tasks/volumes/{overrides', 'tasks/workunit/snaps}', 'tasks/{0-check-counter', 'tasks/{0-from/pacific', 'tasks/{1-thrash/osd', 'test/clone}}', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}', 'workunit/suites/fsstress}', 'wsync/{no}}'] 2022-02-03T20:47:11.913 INFO:scrape: 2022-02-03T20:47:11.913 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi059.front.sepia.ceph.com: ['type=AVC msg=audit(1643893576.556:2657): avc: denied { setgid } for pid=9766 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893610.847:2672): avc: denied { setgid } for pid=10494 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893609.336:2668): avc: denied { setgid } for pid=10482 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893611.356:2674): avc: denied { setgid } for pid=10505 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893578.393:2659): avc: denied { setgid } for pid=9774 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] 2022-02-03T20:47:11.914 INFO:scrape:2 jobs: ['6657886', '6657653'] 2022-02-03T20:47:11.914 INFO:scrape:suites intersection: ['conf/{client', 'distro/{centos_8}', 'mds', 'mon', 'osd}', 'overrides/{frag', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.915 INFO:scrape:suites union: ['2-workunit/suites/ffsb}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'msgr-failures/none', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/fsstress}', 'wsync/{no}}'] 2022-02-03T20:47:11.916 INFO:scrape: 2022-02-03T20:47:11.916 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2022-02-03T20:47:11.917 INFO:scrape:3 jobs: ['6657698', '6658009', '6658004'] 2022-02-03T20:47:11.917 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag', 'races', 'session_timeout', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.918 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', '2-workunit/suites/fsstress}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.918 INFO:scrape: 2022-02-03T20:47:11.919 INFO:scrape:Failure: "2022-02-03T14:39:12.331618+0000 mds.b (mds.0) 1 : cluster [WRN] client.4625 isn't responding to mclientcaps(revoke), ino 0x10000009d13 pending pAsLsXsFsc issued pAsLsXsFscb, sent 300.014114 seconds ago" in cluster log 2022-02-03T20:47:11.919 INFO:scrape:2 jobs: ['6657898', '6657863'] 2022-02-03T20:47:11.920 INFO:scrape:suites intersection: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.920 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.921 INFO:scrape: 2022-02-03T20:47:11.921 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/suites/ffsb.sh 2022-02-03T20:47:11.922 INFO:scrape:2 jobs: ['6657805', '6657899'] 2022-02-03T20:47:11.922 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'ranks/1', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.922 INFO:scrape:suites union: ['2-workunit/suites/ffsb}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'msgr-failures/none', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'races', 'ranks/1', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/ffsb}', 'wsync/{no}}'] 2022-02-03T20:47:11.923 INFO:scrape: 2022-02-03T20:47:11.923 INFO:scrape:Failure: Failure object was: {'smithi148.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:05.023053', 'end': '2022-02-03 11:41:49.863189', 'delta': '0:08:44.840136', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi153.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:05.161656', 'end': '2022-02-03 11:41:51.167671', 'delta': '0:08:46.006015', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') 2022-02-03T20:47:11.924 INFO:scrape:3 jobs: ['6657746', '6657770', '6657747'] 2022-02-03T20:47:11.925 INFO:scrape:suites intersection: ['conf/{client', 'distro/{centos_8}', 'mds', 'mon', 'mount/fuse', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.925 INFO:scrape:suites union: ['2-workunit/suites/ffsb}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'ms_mode/{crc}', 'msgr-failures/none', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag', 'overrides/{no_client_pidfile', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/1', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/snapshots}', 'tasks/{0-check-counter', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/pjd}', 'wsync/{no}}'] 2022-02-03T20:47:11.925 INFO:scrape: 2022-02-03T20:47:11.926 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi166.front.sepia.ceph.com: ['type=AVC msg=audit(1643889975.019:2664): avc: denied { setgid } for pid=9800 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889973.231:2662): avc: denied { setgid } for pid=9792 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] 2022-02-03T20:47:11.926 INFO:scrape:2 jobs: ['6657809', '6657951'] 2022-02-03T20:47:11.927 INFO:scrape:suites intersection: ['1-client', '2-upgrade', 'centos_latest', 'clusters/1-mds-2-client-micro', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{multimds/yes', 'pg-warn', 'tasks/{0-octopus', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.927 INFO:scrape:suites union: ['1-client', '2-upgrade', '3-client-upgrade', '3-compat_client/pacific}}', '4-compat_client', '5-client-sanity}}', 'centos_latest', 'clusters/1-mds-2-client-micro', 'conf/{client', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{multimds/yes', 'pg-warn', 'tasks/{0-octopus', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.928 INFO:scrape: 2022-02-03T20:47:11.928 INFO:scrape:Failure: "2022-02-03T14:03:53.578027+0000 mds.l (mds.2) 324 : cluster [WRN] Scrub error on inode 0x10000001cb7 (/client.0/tmp/t/linux-5.4/Documentation/spi) see mds.l log and `damage ls` output for details" in cluster log 2022-02-03T20:47:11.929 INFO:scrape:2 jobs: ['6657932', '6657658'] 2022-02-03T20:47:11.929 INFO:scrape:suites intersection: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag', 'ranks/3', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'wsync/{no}}'] 2022-02-03T20:47:11.930 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'ranks/3', 'rhel_8}', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}', 'workunit/suites/dbench}', 'wsync/{no}}'] 2022-02-03T20:47:11.930 INFO:scrape: 2022-02-03T20:47:11.931 INFO:scrape:Failure: Command failed (workunit test fs/misc/rstats.sh) on smithi041 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' 2022-02-03T20:47:11.931 INFO:scrape:4 jobs: ['6657927', '6657645', '6657998', '6657783'] 2022-02-03T20:47:11.932 INFO:scrape:suites intersection: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}'] 2022-02-03T20:47:11.932 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-02-03T20:47:11.933 INFO:scrape: 2022-02-03T20:47:11.933 INFO:scrape:Failure: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2022-02-03T20:47:11.933 INFO:scrape:1 jobs: ['6657872'] 2022-02-03T20:47:11.934 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{no_client_pidfile', 'tasks/volumes/{overrides', 'test/misc}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.935 INFO:scrape: 2022-02-03T20:47:11.935 INFO:scrape:Failure: Command failed on smithi008 with status 1: "(cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs)" 2022-02-03T20:47:11.936 INFO:scrape:1 jobs: ['6657860'] 2022-02-03T20:47:11.937 INFO:scrape:suites: ['2-workunit/suites/iozone}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.937 INFO:scrape: 2022-02-03T20:47:11.938 INFO:scrape:Failure: "2022-02-03T15:05:55.342078+0000 mon.a (mon.0) 1493 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log 2022-02-03T20:47:11.938 INFO:scrape:2 jobs: ['6657999', '6657735'] 2022-02-03T20:47:11.938 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{no_client_pidfile', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.939 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/metrics}', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.939 INFO:scrape: 2022-02-03T20:47:11.940 INFO:scrape:Failure: Command failed on smithi136 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-fuse cephfs-java libcephfs_jni1 libcephfs1 librados2 librbd1 python-ceph rbd-fuse python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 2022-02-03T20:47:11.940 INFO:scrape:1 jobs: ['6657800'] 2022-02-03T20:47:11.941 INFO:scrape:suites: ['2-workunit/fs/trivial_sync}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.941 INFO:scrape: 2022-02-03T20:47:11.942 INFO:scrape:Dead: b'2022-02-03T19:05:19.814 DEBUG:teuthology.exit:Finished running handlers' 2022-02-03T20:47:11.942 INFO:scrape:1 jobs: ['6657861'] 2022-02-03T20:47:11.943 INFO:scrape:suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}', 'wsync/{no}}'] 2022-02-03T20:47:11.943 INFO:scrape: 2022-02-03T20:47:11.943 INFO:scrape:Failure: Command failed on smithi110 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-10531-g09fc2e9e-1focal ceph-mds=17.0.0-10531-g09fc2e9e-1focal ceph-common=17.0.0-10531-g09fc2e9e-1focal ceph-fuse=17.0.0-10531-g09fc2e9e-1focal ceph-test=17.0.0-10531-g09fc2e9e-1focal radosgw=17.0.0-10531-g09fc2e9e-1focal python-ceph=17.0.0-10531-g09fc2e9e-1focal libcephfs1=17.0.0-10531-g09fc2e9e-1focal libcephfs-java=17.0.0-10531-g09fc2e9e-1focal libcephfs-jni=17.0.0-10531-g09fc2e9e-1focal librados2=17.0.0-10531-g09fc2e9e-1focal librbd1=17.0.0-10531-g09fc2e9e-1focal rbd-fuse=17.0.0-10531-g09fc2e9e-1focal python3-cephfs=17.0.0-10531-g09fc2e9e-1focal cephfs-shell=17.0.0-10531-g09fc2e9e-1focal cephfs-top=17.0.0-10531-g09fc2e9e-1focal cephfs-mirror=17.0.0-10531-g09fc2e9e-1focal' 2022-02-03T20:47:11.944 INFO:scrape:2 jobs: ['6657797', '6657795'] 2022-02-03T20:47:11.944 INFO:scrape:suites intersection: ['conf/{client', 'distro/{ubuntu_latest}', 'mds', 'mon', 'mount/fuse', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.945 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/32bits/{begin', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{faked-ino', 'overrides/{frag', 'ranks/3', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/cfuse_workunit_suites_pjd}', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}', 'wsync/{yes}}'] 2022-02-03T20:47:11.945 INFO:scrape: 2022-02-03T20:47:11.946 INFO:scrape:Failure: Test failure: test_fuse_mount_on_already_mounted_path (tasks.cephfs.test_misc.TestMisc) 2022-02-03T20:47:11.947 INFO:scrape:2 jobs: ['6657630', '6657849'] 2022-02-03T20:47:11.947 INFO:scrape:suites intersection: ['conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{whitelist_health', 'tasks/cephfs_misc_tests}', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.948 INFO:scrape:suites union: ['clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{whitelist_health', 'tasks/cephfs_misc_tests}', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.948 INFO:scrape: 2022-02-03T20:47:11.949 INFO:scrape:Failure: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2022-02-03T20:47:11.949 INFO:scrape:1 jobs: ['6657683'] 2022-02-03T20:47:11.949 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{no_client_pidfile', 'tasks/pool-perm}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.950 INFO:scrape: 2022-02-03T20:47:11.950 INFO:scrape:Crash: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2022-02-03T20:47:11.951 INFO:scrape:ceph version 17.0.0-10531-g09fc2e9e (09fc2e9e470435f1e6dbf462a51853391f4a4cc0) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fd2589f5b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fd259a11232] 5: /usr/lib64/ceph/libceph-common.so.2(+0x2833f5) [0x7fd259a113f5] 6: (CDir::fetch_keys(std::vector > const&, MDSContext*)+0x43c) [0x558cc765001c] 7: (MDCache::open_undef_inodes_dirfrags()+0x6d6) [0x558cc7531396] 8: (MDCache::rejoin_gather_finish()+0xa8) [0x558cc753e948] 9: (MDCache::handle_cache_rejoin_strong(boost::intrusive_ptr const&)+0x30f1) [0x558cc754a5a1] 10: (MDCache::handle_cache_rejoin(boost::intrusive_ptr const&)+0xdb) [0x558cc754e82b] 11: (MDCache::dispatch(boost::intrusive_ptr const&)+0x354) [0x558cc754ed34] 12: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x942) [0x558cc73de352] 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7cb) [0x558cc73e139b] 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558cc73e19bc] 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x108) [0x558cc73d08b8] 16: (DispatchQueue::entry()+0x14fa) [0x7fd259c9740a] 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd259d4d891] 18: /lib64/libpthread.so.0(+0x814a) [0x7fd2589eb14a] 19: clone() 2022-02-03T20:47:11.951 INFO:scrape:2 jobs: ['6657643', '6657781'] 2022-02-03T20:47:11.952 INFO:scrape:suites intersection: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'races', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.952 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'races', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.953 INFO:scrape: 2022-02-03T20:47:11.953 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2022-02-03T20:47:11.954 INFO:scrape:1 jobs: ['6658005'] 2022-02-03T20:47:11.954 INFO:scrape:suites: ['2-workunit/suites/ffsb}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.954 INFO:scrape: 2022-02-03T20:47:11.955 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi046 with status 141: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 2022-02-03T20:47:11.956 INFO:scrape:1 jobs: ['6657847'] 2022-02-03T20:47:11.957 INFO:scrape:suites: ['clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/libcephfs/{begin', 'mds', 'mon', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{no_client_pidfile', 'tasks/libcephfs/{frag', 'test}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-02-03T20:47:11.957 INFO:scrape: 2022-02-03T20:47:11.958 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2022-02-03T20:47:11.958 INFO:scrape:1 jobs: ['6657639'] 2022-02-03T20:47:11.959 INFO:scrape:suites: ['centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}'] 2022-02-03T20:47:11.959 INFO:scrape: 2022-02-03T20:47:11.997 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/ failed: 182 dead: 58 running: 0 waiting: 0 queued: 0 passed: 196 Fail ================================================================= [6657622] fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137} ----------------------------------------------------------------- time: 00:10:14 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657622/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657622/ SELinux denials found on ubuntu@smithi171.front.sepia.ceph.com: ['type=AVC msg=audit(1643881426.367:2713): avc: denied { setgid } for pid=9967 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881458.000:2726): avc: denied { setgid } for pid=10690 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881424.619:2711): avc: denied { setgid } for pid=9959 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881456.622:2722): avc: denied { setgid } for pid=10683 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881458.468:2728): avc: denied { setgid } for pid=10704 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657623] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} ----------------------------------------------------------------- time: 00:43:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657623/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657623/ SELinux denials found on ubuntu@smithi070.front.sepia.ceph.com: ['type=AVC msg=audit(1643881482.683:2811): avc: denied { setgid } for pid=12264 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881484.562:2816): avc: denied { setgid } for pid=12285 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881417.490:2680): avc: denied { setgid } for pid=9796 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881419.396:2686): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881418.935:2684): avc: denied { setgid } for pid=9805 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881484.990:2817): avc: denied { setgid } for pid=12290 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657626] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} ----------------------------------------------------------------- time: 00:09:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657626/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657626/ SELinux denials found on ubuntu@smithi100.front.sepia.ceph.com: ['type=AVC msg=audit(1643881466.560:2723): avc: denied { setgid } for pid=10736 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881434.695:2715): avc: denied { setgid } for pid=10027 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881434.187:2713): avc: denied { setgid } for pid=10015 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881464.731:2721): avc: denied { setgid } for pid=10727 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881432.782:2709): avc: denied { setgid } for pid=9999 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657628] fs/mirror-ha/{begin cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} ----------------------------------------------------------------- time: 00:26:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657628/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657628/ SELinux denials found on ubuntu@smithi052.front.sepia.ceph.com: ['type=AVC msg=audit(1643881482.534:2686): avc: denied { setgid } for pid=10538 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881448.881:2670): avc: denied { setgid } for pid=9799 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881480.630:2681): avc: denied { setgid } for pid=10517 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881447.489:2669): avc: denied { setgid } for pid=9797 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881449.335:2672): avc: denied { setgid } for pid=9805 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657630] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:21:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657630/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657630/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4236c0251d2944a8888df5508889480f Test failure: test_fuse_mount_on_already_mounted_path (tasks.cephfs.test_misc.TestMisc) [6657635] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:28:57 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657635/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657635/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7fdd265b36f24155a57cbfea1983be80 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6657638] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:27:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657638/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657638/ SELinux denials found on ubuntu@smithi039.front.sepia.ceph.com: ['type=AVC msg=audit(1643881771.976:2667): avc: denied { setgid } for pid=9769 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881772.478:2669): avc: denied { setgid } for pid=9781 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881770.597:2663): avc: denied { setgid } for pid=9761 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657639] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} ----------------------------------------------------------------- time: 01:26:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657639/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657639/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6f8f5361276740ef82741c311f62dcdf Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) [6657641] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:21:40 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657641/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657641/ SELinux denials found on ubuntu@smithi105.front.sepia.ceph.com: ['type=AVC msg=audit(1643881838.054:2688): avc: denied { setgid } for pid=10570 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881803.940:2672): avc: denied { setgid } for pid=9833 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881836.210:2683): avc: denied { setgid } for pid=10549 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881805.748:2677): avc: denied { setgid } for pid=9853 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657643] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:40:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657643/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657643/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c149d5e8639647e6934dba5664d2210f Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6657644] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:03:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657644/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657644/ SELinux denials found on ubuntu@smithi153.front.sepia.ceph.com: ['type=AVC msg=audit(1643881805.451:2662): avc: denied { setgid } for pid=9797 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881806.804:2666): avc: denied { setgid } for pid=9804 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881807.281:2668): avc: denied { setgid } for pid=9817 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881839.484:2679): avc: denied { setgid } for pid=10545 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881837.614:2674): avc: denied { setgid } for pid=10524 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657645] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 00:42:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657645/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657645/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aaea022d7d8e44ada8e2855c152a0142 Command failed (workunit test fs/misc/rstats.sh) on smithi029 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' [6657649] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} ----------------------------------------------------------------- time: 00:11:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657649/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657649/ SELinux denials found on ubuntu@smithi018.front.sepia.ceph.com: ['type=AVC msg=audit(1643881928.900:2675): avc: denied { setgid } for pid=10504 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881893.888:2659): avc: denied { setgid } for pid=9771 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881895.739:2667): avc: denied { setgid } for pid=9791 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643881927.014:2673): avc: denied { setgid } for pid=10495 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657651] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657651/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657651/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7884ead115204c9ea9c76ccff9654753 Command failed (workunit test suites/pjd.sh) on smithi059 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657653] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:58:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657653/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657653/ SELinux denials found on ubuntu@smithi057.front.sepia.ceph.com: ['type=AVC msg=audit(1643882126.266:2687): avc: denied { setgid } for pid=10523 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882124.405:2685): avc: denied { setgid } for pid=10514 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882092.267:2677): avc: denied { setgid } for pid=9802 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882092.747:2679): avc: denied { setgid } for pid=9814 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882090.766:2673): avc: denied { setgid } for pid=9794 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657656] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657656/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657656/ SELinux denials found on ubuntu@smithi103.front.sepia.ceph.com: ['type=AVC msg=audit(1643882144.472:2658): avc: denied { setgid } for pid=9796 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882146.270:2663): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882178.565:2675): avc: denied { setgid } for pid=10540 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882178.051:2673): avc: denied { setgid } for pid=10526 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882176.679:2669): avc: denied { setgid } for pid=10519 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657658] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} ----------------------------------------------------------------- time: 00:49:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657658/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657658/ "2022-02-03T10:10:41.578533+0000 mds.l (mds.0) 99 : cluster [WRN] Scrub error on inode 0x10000000236 (/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.l log and `damage ls` output for details" in cluster log [6657662] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} ----------------------------------------------------------------- time: 01:27:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657662/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657662/ SELinux denials found on ubuntu@smithi017.front.sepia.ceph.com: ['type=AVC msg=audit(1643882407.970:2660): avc: denied { setgid } for pid=9772 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882444.026:2676): avc: denied { setgid } for pid=10509 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882409.825:2662): avc: denied { setgid } for pid=9780 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882442.173:2671): avc: denied { setgid } for pid=10488 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657663] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:57:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657663/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657663/ SELinux denials found on ubuntu@smithi026.front.sepia.ceph.com: ['type=AVC msg=audit(1643882353.362:2674): avc: denied { setgid } for pid=9783 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882388.056:2690): avc: denied { setgid } for pid=10528 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882386.195:2685): avc: denied { setgid } for pid=10507 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882355.176:2679): avc: denied { setgid } for pid=9803 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882388.475:2691): avc: denied { setgid } for pid=10533 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657665] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:31:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657665/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657665/ SELinux denials found on ubuntu@smithi161.front.sepia.ceph.com: ['type=AVC msg=audit(1643882560.286:2837): avc: denied { setgid } for pid=12850 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882482.913:2706): avc: denied { setgid } for pid=11024 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882484.787:2711): avc: denied { setgid } for pid=11044 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882562.190:2842): avc: denied { setgid } for pid=12871 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657668] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} ----------------------------------------------------------------- time: 00:12:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657668/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657668/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2a3902b461e44eeb9a8fc992cbf94249 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [6657670] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657670/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657670/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4b68157d7b3b400a927cbc5183a59583 Command failed (workunit test suites/pjd.sh) on smithi104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657672] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:47:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657672/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657672/ sentry: https://sentry.ceph.com/organizations/ceph/?query=837a692a1225429393c70a90d164e3fd Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6657673] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:51:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657673/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657673/ SELinux denials found on ubuntu@smithi025.front.sepia.ceph.com: ['type=AVC msg=audit(1643883010.557:2686): avc: denied { setgid } for pid=10553 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883008.738:2680): avc: denied { setgid } for pid=10532 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882977.633:2669): avc: denied { setgid } for pid=9819 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883010.081:2684): avc: denied { setgid } for pid=10539 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643882975.797:2667): avc: denied { setgid } for pid=9811 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657677] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657677/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657677/ SELinux denials found on ubuntu@smithi106.front.sepia.ceph.com: ['type=AVC msg=audit(1643883125.768:2668): avc: denied { setgid } for pid=9822 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883123.952:2660): avc: denied { setgid } for pid=9802 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883157.826:2680): avc: denied { setgid } for pid=10547 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883155.933:2674): avc: denied { setgid } for pid=10526 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883157.297:2678): avc: denied { setgid } for pid=10533 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657680] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} ----------------------------------------------------------------- time: 00:11:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657680/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657680/ SELinux denials found on ubuntu@smithi060.front.sepia.ceph.com: ['type=AVC msg=audit(1643883294.724:2674): avc: denied { setgid } for pid=10480 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883261.392:2660): avc: denied { setgid } for pid=9756 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883296.670:2682): avc: denied { setgid } for pid=10501 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883263.262:2665): avc: denied { setgid } for pid=9776 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657683] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} ----------------------------------------------------------------- time: 00:09:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657683/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657683/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0056587001c94f1ca4fa7c26b917f04f Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) [6657685] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:05:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657685/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657685/ SELinux denials found on ubuntu@smithi103.front.sepia.ceph.com: ['type=AVC msg=audit(1643883563.234:2676): avc: denied { setgid } for pid=10528 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883561.872:2672): avc: denied { setgid } for pid=10521 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883563.732:2678): avc: denied { setgid } for pid=10542 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883531.880:2666): avc: denied { setgid } for pid=9825 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883530.053:2658): avc: denied { setgid } for pid=9805 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657688] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:12:46 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657688/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657688/ SELinux denials found on ubuntu@smithi072.front.sepia.ceph.com: ['type=AVC msg=audit(1643883567.842:2677): avc: denied { setgid } for pid=10532 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883534.305:2663): avc: denied { setgid } for pid=9808 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883569.597:2683): avc: denied { setgid } for pid=10553 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883536.038:2668): avc: denied { setgid } for pid=9828 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883569.149:2681): avc: denied { setgid } for pid=10539 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657690] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} ----------------------------------------------------------------- time: 00:57:56 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657690/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657690/ SELinux denials found on ubuntu@smithi037.front.sepia.ceph.com: ['type=AVC msg=audit(1643883589.666:2683): avc: denied { setgid } for pid=10503 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883554.800:2664): avc: denied { setgid } for pid=9758 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883587.746:2678): avc: denied { setgid } for pid=10482 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883556.656:2666): avc: denied { setgid } for pid=9774 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657692] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} ----------------------------------------------------------------- time: 00:08:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657692/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657692/ SELinux denials found on ubuntu@smithi197.front.sepia.ceph.com: ['type=AVC msg=audit(1643883586.298:2662): avc: denied { setgid } for pid=9760 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643883588.100:2667): avc: denied { setgid } for pid=9780 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657693] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657693/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657693/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fbe7977815144ecb6501c425d77726e Command failed (workunit test suites/pjd.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657698] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:22:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657698/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9782c9258d84b7cb9c25713b6968254 reached maximum tries (90) after waiting for 540 seconds [6657699] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:12:21 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657699/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657699/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b196a5527504d85b5e1198f241a433c Command failed (workunit test suites/pjd.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657700] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657700/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657700/ SELinux denials found on ubuntu@smithi040.front.sepia.ceph.com: ['type=AVC msg=audit(1643884118.645:2659): avc: denied { setgid } for pid=9766 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884151.738:2670): avc: denied { setgid } for pid=10472 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884149.860:2668): avc: denied { setgid } for pid=10463 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884116.830:2654): avc: denied { setgid } for pid=9746 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657702] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:58 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657702/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657702/ SELinux denials found on ubuntu@smithi096.front.sepia.ceph.com: ['type=AVC msg=audit(1643884182.836:2671): avc: denied { setgid } for pid=9836 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884216.324:2690): avc: denied { setgid } for pid=10582 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884184.646:2677): avc: denied { setgid } for pid=9856 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884214.430:2685): avc: denied { setgid } for pid=10561 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884184.194:2675): avc: denied { setgid } for pid=9844 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657704] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:54:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657704/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657704/ SELinux denials found on ubuntu@smithi109.front.sepia.ceph.com: ['type=AVC msg=audit(1643884250.192:2673): avc: denied { setgid } for pid=10538 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884248.340:2671): avc: denied { setgid } for pid=10529 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884218.185:2665): avc: denied { setgid } for pid=9821 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884216.375:2659): avc: denied { setgid } for pid=9801 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643884217.712:2663): avc: denied { setgid } for pid=9807 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657706] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:32:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657706/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657706/ sentry: https://sentry.ceph.com/organizations/ceph/?query=685d82c20ccf4365beb8d7fc8d980a81 Test failure: test_standby (tasks.cephfs.test_failover.TestMultiFilesystems) [6657710] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:09:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657710/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657710/ SELinux denials found on ubuntu@smithi080.front.sepia.ceph.com: ['type=AVC msg=audit(1643886877.542:2700): avc: denied { setgid } for pid=9951 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643886911.409:2713): avc: denied { setgid } for pid=10677 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643886879.317:2702): avc: denied { setgid } for pid=9959 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643886909.622:2711): avc: denied { setgid } for pid=10668 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657735] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:30:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657735/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657735/ "2022-02-03T11:35:27.913180+0000 mon.a (mon.0) 416 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6657778] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:20:03 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657778/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657778/ sentry: https://sentry.ceph.com/organizations/ceph/?query=076814940c104f679a0efef2597b16ad No module named 'tasks.fs' [6657781] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:48:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657781/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657781/ sentry: https://sentry.ceph.com/organizations/ceph/?query=731d883791f94a838d9d388d5f34553d Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi197 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6657782] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 02:10:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657782/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657782/ SELinux denials found on ubuntu@smithi149.front.sepia.ceph.com: ['type=AVC msg=audit(1643889375.830:2669): avc: denied { setgid } for pid=10532 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889340.849:2653): avc: denied { setgid } for pid=9786 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889342.614:2658): avc: denied { setgid } for pid=9806 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889373.993:2664): avc: denied { setgid } for pid=10503 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657783] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} ----------------------------------------------------------------- time: 00:28:48 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657783/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657783/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3bfdb139243a45faaba5758d0c4f7864 Command failed (workunit test fs/misc/rstats.sh) on smithi143 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' [6657788] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} ----------------------------------------------------------------- time: 00:13:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657788/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657788/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69e5fa5f81a241fc9a82d29e1c11ae5d No module named 'tasks.kclient' [6657789] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:03 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657789/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657789/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d41fb2f7ee834c07909e82cc30b37308 Command failed (workunit test suites/pjd.sh) on smithi148 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657790] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:16:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657790/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657790/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7101a07260b046b4a6b4f158c7158b53 No module named 'tasks.kclient' [6657792] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:11:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657792/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657792/ sentry: https://sentry.ceph.com/organizations/ceph/?query=43b1ab96a4684e94b4f774e3a6e1b1f9 No module named 'tasks.ceph' [6657793] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:07:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657793/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657793/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a22601a23aab44f58bd49f61283e04da No module named 'tasks.ceph' [6657795] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} ----------------------------------------------------------------- time: 00:04:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657795/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657795/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e65930d1d16427ea2ccf455e298f5cf Command failed on smithi161 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::=" --force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-10531-g09fc2e9e-1focal ceph-mds=17.0.0-10531-g09fc2e9e-1focal ceph-common=17.0.0-10531-g09fc2e9e-1focal ceph- fuse=17.0.0-10531-g09fc2e9e-1focal ceph-test=17.0.0-10531-g09fc2e9e-1focal radosgw=17.0.0-10531-g09fc2e9e-1focal python- ceph=17.0.0-10531-g09fc2e9e-1focal libcephfs1=17.0.0-10531-g09fc2e9e-1focal libcephfs-java=17.0.0-10531-g09fc2e9e-1focal libcephfs- jni=17.0.0-10531-g09fc2e9e-1focal librados2=17.0.0-10531-g09fc2e9e-1focal librbd1=17.0.0-10531-g09fc2e9e-1focal rbd- fuse=17.0.0-10531-g09fc2e9e-1focal python3-cephfs=17.0.0-10531-g09fc2e9e-1focal cephfs- shell=17.0.0-10531-g09fc2e9e-1focal cephfs- top=17.0.0-10531-g09fc2e9e-1focal cephfs- mirror=17.0.0-10531-g09fc2e9e-1focal' [6657797] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:04:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657797/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657797/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ade5ecd95bf4442db3dd9c7ddaf0f042 Command failed on smithi110 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::=" --force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-10531-g09fc2e9e-1focal ceph-mds=17.0.0-10531-g09fc2e9e-1focal ceph-common=17.0.0-10531-g09fc2e9e-1focal ceph- fuse=17.0.0-10531-g09fc2e9e-1focal ceph-test=17.0.0-10531-g09fc2e9e-1focal radosgw=17.0.0-10531-g09fc2e9e-1focal python- ceph=17.0.0-10531-g09fc2e9e-1focal libcephfs1=17.0.0-10531-g09fc2e9e-1focal libcephfs-java=17.0.0-10531-g09fc2e9e-1focal libcephfs- jni=17.0.0-10531-g09fc2e9e-1focal librados2=17.0.0-10531-g09fc2e9e-1focal librbd1=17.0.0-10531-g09fc2e9e-1focal rbd- fuse=17.0.0-10531-g09fc2e9e-1focal python3-cephfs=17.0.0-10531-g09fc2e9e-1focal cephfs- shell=17.0.0-10531-g09fc2e9e-1focal cephfs- top=17.0.0-10531-g09fc2e9e-1focal cephfs- mirror=17.0.0-10531-g09fc2e9e-1focal' [6657800] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:03:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657800/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657800/ sentry: https://sentry.ceph.com/organizations/ceph/?query=85c785980cce4da1b08f66d9c714b60f Command failed on smithi136 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-fuse cephfs-java libcephfs_jni1 libcephfs1 librados2 librbd1 python-ceph rbd-fuse python3-cephfs cephfs-top cephfs- mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid- devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' [6657805] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} ----------------------------------------------------------------- time: 03:11:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657805/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657805/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f1f26cf8d82f4ed9ba7d3e17e076efb9 Command failed (workunit test suites/ffsb.sh) on smithi039 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [6657806] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:42:49 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657806/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657806/ SELinux denials found on ubuntu@smithi029.front.sepia.ceph.com: ['type=AVC msg=audit(1643889966.474:2659): avc: denied { setgid } for pid=9812 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890002.158:2667): avc: denied { setgid } for pid=10536 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889963.388:2653): avc: denied { setgid } for pid=9799 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889966.058:2655): avc: denied { setgid } for pid=9807 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890000.254:2665): avc: denied { setgid } for pid=10527 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657807] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:12:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657807/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657807/ SELinux denials found on ubuntu@smithi031.front.sepia.ceph.com: ['type=AVC msg=audit(1643890052.496:2664): avc: denied { setgid } for pid=9769 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890086.172:2677): avc: denied { setgid } for pid=10510 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890052.976:2666): avc: denied { setgid } for pid=9781 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890084.261:2672): avc: denied { setgid } for pid=10489 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890051.120:2660): avc: denied { setgid } for pid=9761 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657809] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:20:14 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657809/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657809/ SELinux denials found on ubuntu@smithi166.front.sepia.ceph.com: ['type=AVC msg=audit(1643889975.019:2664): avc: denied { setgid } for pid=9800 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643889973.231:2662): avc: denied { setgid } for pid=9792 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657811] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} ----------------------------------------------------------------- time: 00:12:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657811/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657811/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0ee4919b11194b65a0e7b4820ae0f903 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [6657813] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657813/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657813/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c069590f9576492f8ea17f51e3d0c5bf Command failed (workunit test suites/pjd.sh) on smithi079 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657814] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} ----------------------------------------------------------------- time: 00:07:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657814/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657814/ sentry: https://sentry.ceph.com/organizations/ceph/?query=302e262c352d44fd9b2e3fb9591028eb No module named 'tasks' [6657815] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:39:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657815/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657815/ sentry: https://sentry.ceph.com/organizations/ceph/?query=81c66a3897324aa59e94a4362372b470 Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi033 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git- ceph.sh' [6657816] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:51:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657816/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657816/ SELinux denials found on ubuntu@smithi025.front.sepia.ceph.com: ['type=AVC msg=audit(1643890252.904:2672): avc: denied { setgid } for pid=9819 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890251.070:2670): avc: denied { setgid } for pid=9811 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890286.013:2686): avc: denied { setgid } for pid=10556 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890284.101:2681): avc: denied { setgid } for pid=10535 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657817] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} ----------------------------------------------------------------- time: 00:45:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657817/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657817/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cbda4ecfa0264c308f3d4a86e9681c77 Test failure: test_mirroring_with_filesystem_reset (tasks.cephfs.test_admin.TestMirroringCommands) [6657820] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:09:48 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657820/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657820/ SELinux denials found on ubuntu@smithi005.front.sepia.ceph.com: ['type=AVC msg=audit(1643890433.467:2662): avc: denied { setgid } for pid=9758 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890465.157:2668): avc: denied { setgid } for pid=10466 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890431.565:2656): avc: denied { setgid } for pid=9738 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890467.026:2673): avc: denied { setgid } for pid=10487 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890432.939:2660): avc: denied { setgid } for pid=9746 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657823] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} ----------------------------------------------------------------- time: 00:11:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657823/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657823/ SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1643890511.719:2652): avc: denied { setgid } for pid=9732 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890545.260:2666): avc: denied { setgid } for pid=10455 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890513.581:2660): avc: denied { setgid } for pid=9752 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890547.116:2668): avc: denied { setgid } for pid=10464 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657825] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} ----------------------------------------------------------------- time: 00:10:14 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657825/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657825/ SELinux denials found on ubuntu@smithi161.front.sepia.ceph.com: ['type=AVC msg=audit(1643890689.562:2683): avc: denied { setgid } for pid=10566 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890657.551:2669): avc: denied { setgid } for pid=9828 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890687.650:2678): avc: denied { setgid } for pid=10545 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890655.766:2667): avc: denied { setgid } for pid=9820 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657828] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:59:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657828/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657828/ SELinux denials found on ubuntu@smithi148.front.sepia.ceph.com: ['type=AVC msg=audit(1643890780.512:2666): avc: denied { setgid } for pid=10506 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890749.148:2656): avc: denied { setgid } for pid=9795 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890779.128:2662): avc: denied { setgid } for pid=10499 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890780.971:2668): avc: denied { setgid } for pid=10520 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890747.385:2651): avc: denied { setgid } for pid=9775 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657833] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:32:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657833/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657833/ SELinux denials found on ubuntu@smithi099.front.sepia.ceph.com: ['type=AVC msg=audit(1643890943.755:2862): avc: denied { setgid } for pid=12892 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890876.529:2722): avc: denied { setgid } for pid=11045 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890878.293:2724): avc: denied { setgid } for pid=11053 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643890941.885:2857): avc: denied { setgid } for pid=12871 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657835] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657835/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657835/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16d8c99392bd4a46bfd5fdbbe2e4907c Command failed (workunit test suites/pjd.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657840] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:11:57 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657840/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657840/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4a697a087bba4c4891bfadf7f5078742 Command failed (workunit test suites/pjd.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657841] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657841/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657841/ SELinux denials found on ubuntu@smithi055.front.sepia.ceph.com: ['type=AVC msg=audit(1643891511.738:2670): avc: denied { setgid } for pid=10476 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891478.794:2659): avc: denied { setgid } for pid=9760 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891513.649:2672): avc: denied { setgid } for pid=10485 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891480.614:2664): avc: denied { setgid } for pid=9780 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657843] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} ----------------------------------------------------------------- time: 00:09:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657843/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657843/ SELinux denials found on ubuntu@smithi093.front.sepia.ceph.com: ['type=AVC msg=audit(1643891611.692:2672): avc: denied { setgid } for pid=10554 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891609.890:2670): avc: denied { setgid } for pid=10545 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891577.277:2659): avc: denied { setgid } for pid=9821 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891579.078:2664): avc: denied { setgid } for pid=9841 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657844] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657844/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657844/ SELinux denials found on ubuntu@smithi079.front.sepia.ceph.com: ['type=AVC msg=audit(1643891575.494:2669): avc: denied { setgid } for pid=9821 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891606.926:2680): avc: denied { setgid } for pid=10545 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891609.168:2686): avc: denied { setgid } for pid=10571 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891608.770:2685): avc: denied { setgid } for pid=10566 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891577.252:2674): avc: denied { setgid } for pid=9841 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657845] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657845/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657845/ SELinux denials found on ubuntu@smithi064.front.sepia.ceph.com: ['type=AVC msg=audit(1643891610.238:2672): avc: denied { setgid } for pid=9832 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891641.606:2681): avc: denied { setgid } for pid=10541 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891608.380:2670): avc: denied { setgid } for pid=9824 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891643.460:2686): avc: denied { setgid } for pid=10562 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657846] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/whitelist_health} ----------------------------------------------------------------- time: 00:16:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657846/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657846/ SELinux denials found on ubuntu@smithi124.front.sepia.ceph.com: ['type=AVC msg=audit(1643891556.333:2669): avc: denied { setgid } for pid=9790 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891622.285:2799): avc: denied { setgid } for pid=12238 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891620.365:2794): avc: denied { setgid } for pid=12217 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891554.508:2664): avc: denied { setgid } for pid=9770 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657847] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}} ----------------------------------------------------------------- time: 00:12:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657847/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657847/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70db3d7907b944debb126b4972fc03ed Command failed (workunit test libcephfs/test.sh) on smithi046 with status 141: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' [6657849] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:20:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657849/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67430d5b4c3543b1b614940e96b405dd Test failure: test_fuse_mount_on_already_mounted_path (tasks.cephfs.test_misc.TestMisc) [6657850] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:21:49 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657850/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657850/ sentry: https://sentry.ceph.com/organizations/ceph/?query=318bd3e5684147cea99feb159d287eba Test failure: test_thrash (tasks.cephfs.test_failover.TestClusterResize) [6657855] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 02:13:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657855/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657855/ SELinux denials found on ubuntu@smithi137.front.sepia.ceph.com: ['type=AVC msg=audit(1643891837.493:2667): avc: denied { setgid } for pid=10501 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891807.284:2658): avc: denied { setgid } for pid=9797 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891805.432:2653): avc: denied { setgid } for pid=9777 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643891839.305:2669): avc: denied { setgid } for pid=10510 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657857] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} ----------------------------------------------------------------- time: 02:05:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657857/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657857/ sentry: https://sentry.ceph.com/organizations/ceph/?query=09b4babfdfef489291a1d57b4202d765 Command failed (workunit test fs/misc/rstats.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' [6657860] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:12:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657860/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657860/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ff04cd917dca4a7780d48d9f5060b559 Command failed on smithi008 with status 1: "(cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest- mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs)" [6657862] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657862/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657862/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1bb9ef0e366f492b9c3294f9f3348c02 Command failed (workunit test suites/pjd.sh) on smithi059 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657863] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 03:31:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657863/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657863/ "2022-02-03T14:20:31.027597+0000 mds.a (mds.3) 1 : cluster [WRN] client.4743 isn't responding to mclientcaps(revoke), ino 0x100000081df pending pAsLsXsFsc issued pAsLsXsFscb, sent 300.014801 seconds ago" in cluster log [6657865] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:27:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657865/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657865/ SELinux denials found on ubuntu@smithi057.front.sepia.ceph.com: ['type=AVC msg=audit(1643892533.097:2665): avc: denied { setgid } for pid=9732 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892531.219:2663): avc: denied { setgid } for pid=9724 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657866] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} ----------------------------------------------------------------- time: 00:16:51 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657866/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657866/ SELinux denials found on ubuntu@smithi140.front.sepia.ceph.com: ['type=AVC msg=audit(1643892668.658:2664): avc: denied { setgid } for pid=9836 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892700.528:2675): avc: denied { setgid } for pid=10561 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892666.811:2659): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892698.600:2670): avc: denied { setgid } for pid=10540 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657869] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657869/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657869/ SELinux denials found on ubuntu@smithi055.front.sepia.ceph.com: ['type=AVC msg=audit(1643892741.789:2670): avc: denied { setgid } for pid=10481 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892706.549:2654): avc: denied { setgid } for pid=9736 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892739.903:2665): avc: denied { setgid } for pid=10460 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892708.387:2659): avc: denied { setgid } for pid=9756 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657870] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:40:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657870/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657870/ SELinux denials found on ubuntu@smithi093.front.sepia.ceph.com: ['type=AVC msg=audit(1643892776.366:2677): avc: denied { setgid } for pid=10557 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892774.571:2675): avc: denied { setgid } for pid=10548 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892743.654:2669): avc: denied { setgid } for pid=9840 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892741.817:2663): avc: denied { setgid } for pid=9820 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892743.160:2667): avc: denied { setgid } for pid=9828 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657872] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:11:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657872/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657872/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cf3b4e58aa974a389c2d42b03b89b657 Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) [6657875] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657875/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657875/ SELinux denials found on ubuntu@smithi046.front.sepia.ceph.com: ['type=AVC msg=audit(1643892965.749:2682): avc: denied { setgid } for pid=10509 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892931.131:2663): avc: denied { setgid } for pid=9773 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892963.869:2677): avc: denied { setgid } for pid=10488 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643892932.985:2668): avc: denied { setgid } for pid=9793 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657879] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 02:11:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657879/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657879/ SELinux denials found on ubuntu@smithi124.front.sepia.ceph.com: ['type=AVC msg=audit(1643893260.763:2709): avc: denied { setgid } for pid=9972 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893292.873:2720): avc: denied { setgid } for pid=10710 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893258.963:2707): avc: denied { setgid } for pid=9964 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893291.153:2718): avc: denied { setgid } for pid=10701 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657882] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} ----------------------------------------------------------------- time: 00:16:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657882/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657882/ sentry: https://sentry.ceph.com/organizations/ceph/?query=049ce74938754cad87033633f0f4dbaa Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [6657883] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657883/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657883/ SELinux denials found on ubuntu@smithi167.front.sepia.ceph.com: ['type=AVC msg=audit(1643893498.631:2676): avc: denied { setgid } for pid=9871 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893530.306:2684): avc: denied { setgid } for pid=10577 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893528.454:2682): avc: denied { setgid } for pid=10568 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893496.810:2671): avc: denied { setgid } for pid=9851 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657885] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:46 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657885/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657885/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ff882b15e7054c108b682b53de444086 Command failed (workunit test suites/pjd.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657886] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} ----------------------------------------------------------------- time: 00:12:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657886/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657886/ SELinux denials found on ubuntu@smithi059.front.sepia.ceph.com: ['type=AVC msg=audit(1643893576.556:2657): avc: denied { setgid } for pid=9766 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893610.847:2672): avc: denied { setgid } for pid=10494 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893609.336:2668): avc: denied { setgid } for pid=10482 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893611.356:2674): avc: denied { setgid } for pid=10505 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893578.393:2659): avc: denied { setgid } for pid=9774 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657887] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:40:49 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657887/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657887/ sentry: https://sentry.ceph.com/organizations/ceph/?query=423d7f52e9554bf79a4eb1ccdd0e7cf2 Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi005 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git- ceph.sh' [6657888] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} ----------------------------------------------------------------- time: 00:09:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657888/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657888/ SELinux denials found on ubuntu@smithi160.front.sepia.ceph.com: ['type=AVC msg=audit(1643893547.068:2665): avc: denied { setgid } for pid=9800 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893578.881:2677): avc: denied { setgid } for pid=10528 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893545.725:2661): avc: denied { setgid } for pid=9793 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893547.541:2667): avc: denied { setgid } for pid=9813 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893577.573:2673): avc: denied { setgid } for pid=10521 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893579.393:2679): avc: denied { setgid } for pid=10542 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657892] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:09:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657892/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657892/ SELinux denials found on ubuntu@smithi018.front.sepia.ceph.com: ['type=AVC msg=audit(1643893656.791:2652): avc: denied { setgid } for pid=9742 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893654.953:2650): avc: denied { setgid } for pid=9734 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893689.875:2663): avc: denied { setgid } for pid=10459 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893688.038:2661): avc: denied { setgid } for pid=10450 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657893] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:33:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657893/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657893/ SELinux denials found on ubuntu@smithi033.front.sepia.ceph.com: ['type=AVC msg=audit(1643893870.993:2906): avc: denied { setgid } for pid=13006 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893870.475:2904): avc: denied { setgid } for pid=12991 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893869.089:2900): avc: denied { setgid } for pid=12985 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893803.363:2724): avc: denied { setgid } for pid=11022 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893801.383:2722): avc: denied { setgid } for pid=11014 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893803.796:2728): avc: denied { setgid } for pid=11027 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657895] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} ----------------------------------------------------------------- time: 00:12:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657895/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657895/ SELinux denials found on ubuntu@smithi017.front.sepia.ceph.com: ['type=AVC msg=audit(1643893982.596:2677): avc: denied { setgid } for pid=9808 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894016.031:2688): avc: denied { setgid } for pid=10533 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894014.113:2683): avc: denied { setgid } for pid=10512 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643893980.661:2672): avc: denied { setgid } for pid=9788 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657897] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:32:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657897/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657897/ "2022-02-03T13:39:22.932742+0000 mon.a (mon.0) 2322 : cluster [WRN] Replacing daemon mds.c as rank 0 with standby daemon mds.a" in cluster log [6657898] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 04:49:56 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657898/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657898/ "2022-02-03T14:39:12.331618+0000 mds.b (mds.0) 1 : cluster [WRN] client.4625 isn't responding to mclientcaps(revoke), ino 0x10000009d13 pending pAsLsXsFsc issued pAsLsXsFscb, sent 300.014114 seconds ago" in cluster log [6657899] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 03:15:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657899/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657899/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd3d831a50164d3ca7aee05146e704a7 Command failed (workunit test suites/ffsb.sh) on smithi132 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [6657902] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:15:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657902/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657902/ SELinux denials found on ubuntu@smithi140.front.sepia.ceph.com: ['type=AVC msg=audit(1643894312.156:2667): avc: denied { setgid } for pid=9822 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894313.919:2669): avc: denied { setgid } for pid=9830 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894346.831:2683): avc: denied { setgid } for pid=10567 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894344.978:2678): avc: denied { setgid } for pid=10546 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657904] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657904/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657904/ SELinux denials found on ubuntu@smithi131.front.sepia.ceph.com: ['type=AVC msg=audit(1643894543.750:2673): avc: denied { setgid } for pid=9828 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894577.811:2690): avc: denied { setgid } for pid=10573 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894577.302:2688): avc: denied { setgid } for pid=10559 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894545.594:2678): avc: denied { setgid } for pid=9848 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894575.957:2684): avc: denied { setgid } for pid=10552 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657905] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} ----------------------------------------------------------------- time: 00:20:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657905/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657905/ SELinux denials found on ubuntu@smithi161.front.sepia.ceph.com: ['type=AVC msg=audit(1643894562.112:2658): avc: denied { setgid } for pid=9787 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894563.986:2664): avc: denied { setgid } for pid=9807 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894593.688:2670): avc: denied { setgid } for pid=10515 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894595.499:2672): avc: denied { setgid } for pid=10524 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894563.474:2662): avc: denied { setgid } for pid=9795 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657907] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} ----------------------------------------------------------------- time: 00:26:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657907/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657907/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2a1aa437633c4878aa90c8b5e2031d61 Test failure: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) [6657908] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:05 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657908/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657908/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1829eb6345e47b492590073d8cadb7f Command failed (workunit test suites/pjd.sh) on smithi103 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657911] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:55:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657911/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657911/ SELinux denials found on ubuntu@smithi077.front.sepia.ceph.com: ['type=AVC msg=audit(1643894796.239:2675): avc: denied { setgid } for pid=10534 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894764.075:2664): avc: denied { setgid } for pid=9810 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894766.018:2669): avc: denied { setgid } for pid=9830 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894798.083:2680): avc: denied { setgid } for pid=10555 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657913] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:09:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657913/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657913/ SELinux denials found on ubuntu@smithi099.front.sepia.ceph.com: ['type=AVC msg=audit(1643894883.008:2677): avc: denied { setgid } for pid=10541 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894848.958:2657): avc: denied { setgid } for pid=9796 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894882.539:2675): avc: denied { setgid } for pid=10527 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894850.761:2662): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894881.154:2671): avc: denied { setgid } for pid=10520 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657914] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} ----------------------------------------------------------------- time: 00:12:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657914/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657914/ SELinux denials found on ubuntu@smithi043.front.sepia.ceph.com: ['type=AVC msg=audit(1643895000.518:2683): avc: denied { setgid } for pid=10516 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894967.225:2669): avc: denied { setgid } for pid=9790 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894998.673:2679): avc: denied { setgid } for pid=10503 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894965.360:2667): avc: denied { setgid } for pid=9782 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643894967.648:2673): avc: denied { setgid } for pid=9795 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657916] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} ----------------------------------------------------------------- time: 00:21:46 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657916/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657916/ sentry: https://sentry.ceph.com/organizations/ceph/?query=701e1459ab3f4f0dbf5f066eb8068dd0 Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDataScan) [6657917] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657917/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657917/ SELinux denials found on ubuntu@smithi062.front.sepia.ceph.com: ['type=AVC msg=audit(1643895043.658:2680): avc: denied { setgid } for pid=10521 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895010.337:2666): avc: denied { setgid } for pid=9797 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895012.455:2668): avc: denied { setgid } for pid=9805 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895045.492:2685): avc: denied { setgid } for pid=10542 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657918] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:27:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657918/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657918/ SELinux denials found on ubuntu@smithi079.front.sepia.ceph.com: ['type=AVC msg=audit(1643895023.790:2661): avc: denied { setgid } for pid=9811 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895022.037:2659): avc: denied { setgid } for pid=9803 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895056.196:2672): avc: denied { setgid } for pid=10536 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895054.401:2670): avc: denied { setgid } for pid=10527 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657920] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:30:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657920/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657920/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a2198f8d8d164343a90640ea01eb6964 Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) [6657921] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:28:41 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657921/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657921/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3cde684990484929be56607cd3ab7edc Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi068 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git- ceph.sh' [6657922] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:30:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657922/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657922/ SELinux denials found on ubuntu@smithi071.front.sepia.ceph.com: ['type=AVC msg=audit(1643895683.706:2837): avc: denied { setgid } for pid=12836 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895615.678:2706): avc: denied { setgid } for pid=11007 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895685.844:2842): avc: denied { setgid } for pid=12859 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895617.511:2711): avc: denied { setgid } for pid=11027 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657924] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} ----------------------------------------------------------------- time: 00:16:33 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657924/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657924/ SELinux denials found on ubuntu@smithi073.front.sepia.ceph.com: ['type=AVC msg=audit(1643895893.559:2688): avc: denied { setgid } for pid=10564 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895859.519:2673): avc: denied { setgid } for pid=9840 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895861.264:2675): avc: denied { setgid } for pid=9848 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895894.080:2690): avc: denied { setgid } for pid=10578 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643895892.226:2684): avc: denied { setgid } for pid=10557 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657927] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} ----------------------------------------------------------------- time: 00:35:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657927/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657927/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f4fa21667927434f9ea103d42d473529 Command failed (workunit test fs/misc/rstats.sh) on smithi041 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' [6657928] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:18:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657928/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657928/ SELinux denials found on ubuntu@smithi082.front.sepia.ceph.com: ['type=AVC msg=audit(1643896046.058:2672): avc: denied { setgid } for pid=10533 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896016.247:2666): avc: denied { setgid } for pid=9826 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896014.442:2660): avc: denied { setgid } for pid=9806 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896047.844:2674): avc: denied { setgid } for pid=10542 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896015.765:2664): avc: denied { setgid } for pid=9812 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657929] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} ----------------------------------------------------------------- time: 00:43:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657929/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657929/ SELinux denials found on ubuntu@smithi099.front.sepia.ceph.com: ['type=AVC msg=audit(1643896040.579:2663): avc: denied { setgid } for pid=9818 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896070.968:2669): avc: denied { setgid } for pid=10522 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896072.778:2671): avc: denied { setgid } for pid=10531 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896038.413:2658): avc: denied { setgid } for pid=9798 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657932] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} ----------------------------------------------------------------- time: 00:38:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657932/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657932/ "2022-02-03T14:03:53.578027+0000 mds.l (mds.2) 324 : cluster [WRN] Scrub error on inode 0x10000001cb7 (/client.0/tmp/t/linux-5.4/Documentation/spi) see mds.l log and `damage ls` output for details" in cluster log [6657933] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:17:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657933/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657933/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee3ac3386de2402c9c386814af9d0ccf Command failed (workunit test suites/pjd.sh) on smithi065 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657941] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:12:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657941/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657941/ SELinux denials found on ubuntu@smithi036.front.sepia.ceph.com: ['type=AVC msg=audit(1643896589.649:2669): avc: denied { setgid } for pid=10480 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896591.611:2674): avc: denied { setgid } for pid=10503 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896556.449:2658): avc: denied { setgid } for pid=9756 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896558.301:2663): avc: denied { setgid } for pid=9776 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657942] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:21 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657942/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657942/ sentry: https://sentry.ceph.com/organizations/ceph/?query=056311b8c5044982b7b258e471e3bfdc Command failed (workunit test suites/pjd.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657943] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:54:07 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657943/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657943/ SELinux denials found on ubuntu@smithi045.front.sepia.ceph.com: ['type=AVC msg=audit(1643896793.236:2681): avc: denied { setgid } for pid=10499 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896760.487:2665): avc: denied { setgid } for pid=9776 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896758.652:2663): avc: denied { setgid } for pid=9768 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896793.719:2683): avc: denied { setgid } for pid=10513 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896791.873:2677): avc: denied { setgid } for pid=10484 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657949] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} ----------------------------------------------------------------- time: 00:18:46 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657949/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657949/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ae11dfc549f43c489376d8a99e7e437 Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation) [6657950] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:19:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657950/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657950/ SELinux denials found on ubuntu@smithi160.front.sepia.ceph.com: ['type=AVC msg=audit(1643896894.757:2666): avc: denied { setgid } for pid=9818 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896893.363:2662): avc: denied { setgid } for pid=9810 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896925.763:2674): avc: denied { setgid } for pid=10537 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896927.600:2676): avc: denied { setgid } for pid=10546 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896895.246:2668): avc: denied { setgid } for pid=9830 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657951] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:19:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657951/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657951/ SELinux denials found on ubuntu@smithi008.front.sepia.ceph.com: ['type=AVC msg=audit(1643896969.245:2670): avc: denied { setgid } for pid=9755 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643896967.325:2665): avc: denied { setgid } for pid=9735 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657953] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} ----------------------------------------------------------------- time: 00:10:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657953/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657953/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f90c3358d4ef44648d8bd32474a64aac Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [6657954] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:13:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657954/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657954/ SELinux denials found on ubuntu@smithi079.front.sepia.ceph.com: ['type=AVC msg=audit(1643897350.126:2673): avc: denied { setgid } for pid=9831 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897349.712:2669): avc: denied { setgid } for pid=9826 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897380.100:2679): avc: denied { setgid } for pid=10546 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897348.001:2667): avc: denied { setgid } for pid=9818 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897381.837:2681): avc: denied { setgid } for pid=10555 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657955] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:17:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657955/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657955/ sentry: https://sentry.ceph.com/organizations/ceph/?query=630349a1c10647cc8b7eafb1fd694a9c Command failed (workunit test suites/pjd.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657956] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657956/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657956/ SELinux denials found on ubuntu@smithi073.front.sepia.ceph.com: ['type=AVC msg=audit(1643897479.348:2681): avc: denied { setgid } for pid=10540 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897446.577:2670): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897481.586:2687): avc: denied { setgid } for pid=10566 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897481.175:2686): avc: denied { setgid } for pid=10561 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897448.399:2675): avc: denied { setgid } for pid=9836 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657957] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} ----------------------------------------------------------------- time: 00:09:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657957/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657957/ SELinux denials found on ubuntu@smithi103.front.sepia.ceph.com: ['type=AVC msg=audit(1643897635.893:2675): avc: denied { setgid } for pid=9843 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897665.895:2684): avc: denied { setgid } for pid=10552 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897667.707:2689): avc: denied { setgid } for pid=10573 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897634.180:2673): avc: denied { setgid } for pid=9835 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657958] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657958/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657958/ SELinux denials found on ubuntu@smithi065.front.sepia.ceph.com: ['type=AVC msg=audit(1643897709.848:2669): avc: denied { setgid } for pid=9815 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897742.538:2683): avc: denied { setgid } for pid=10540 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897707.935:2664): avc: denied { setgid } for pid=9795 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897740.644:2678): avc: denied { setgid } for pid=10519 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657960] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:55:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657960/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657960/ SELinux denials found on ubuntu@smithi082.front.sepia.ceph.com: ['type=AVC msg=audit(1643897745.315:2665): avc: denied { setgid } for pid=9816 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897777.103:2673): avc: denied { setgid } for pid=10529 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897775.267:2671): avc: denied { setgid } for pid=10520 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643897743.452:2657): avc: denied { setgid } for pid=9796 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657964] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} ----------------------------------------------------------------- time: 00:21:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657964/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657964/ SELinux denials found on ubuntu@smithi036.front.sepia.ceph.com: ['type=AVC msg=audit(1643898008.117:2671): avc: denied { setgid } for pid=9793 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898041.714:2686): avc: denied { setgid } for pid=10504 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898039.317:2680): avc: denied { setgid } for pid=10490 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898041.246:2682): avc: denied { setgid } for pid=10499 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898006.295:2666): avc: denied { setgid } for pid=9773 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657966] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:17:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657966/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657966/ SELinux denials found on ubuntu@smithi046.front.sepia.ceph.com: ['type=AVC msg=audit(1643898079.628:2671): avc: denied { setgid } for pid=10488 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898048.645:2665): avc: denied { setgid } for pid=9788 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898046.794:2659): avc: denied { setgid } for pid=9760 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898048.164:2663): avc: denied { setgid } for pid=9775 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898081.490:2676): avc: denied { setgid } for pid=10509 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657967] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657967/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657967/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1d8c54ae29543fab1797431f35f5d70 Command failed (workunit test suites/pjd.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657969] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} ----------------------------------------------------------------- time: 00:09:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657969/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657969/ SELinux denials found on ubuntu@smithi005.front.sepia.ceph.com: ['type=AVC msg=audit(1643898204.473:2658): avc: denied { setgid } for pid=9758 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898202.398:2656): avc: denied { setgid } for pid=9750 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898235.857:2667): avc: denied { setgid } for pid=10466 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898237.683:2672): avc: denied { setgid } for pid=10487 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657972] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} ----------------------------------------------------------------- time: 00:20:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657972/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657972/ SELinux denials found on ubuntu@smithi018.front.sepia.ceph.com: ['type=AVC msg=audit(1643898911.270:2662): avc: denied { setgid } for pid=9773 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898944.214:2673): avc: denied { setgid } for pid=10498 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898942.394:2668): avc: denied { setgid } for pid=10477 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643898909.404:2657): avc: denied { setgid } for pid=9753 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657974] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:16:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657974/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657974/ SELinux denials found on ubuntu@smithi065.front.sepia.ceph.com: ['type=AVC msg=audit(1643899079.014:2662): avc: denied { setgid } for pid=9811 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899110.024:2671): avc: denied { setgid } for pid=10515 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899077.055:2657): avc: denied { setgid } for pid=9791 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899111.901:2673): avc: denied { setgid } for pid=10524 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657979] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:31:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657979/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657979/ SELinux denials found on ubuntu@smithi103.front.sepia.ceph.com: ['type=AVC msg=audit(1643899432.364:2859): avc: denied { setgid } for pid=12903 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899367.772:2724): avc: denied { setgid } for pid=11075 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899365.727:2719): avc: denied { setgid } for pid=11055 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899430.627:2854): avc: denied { setgid } for pid=12882 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657980] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:17:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657980/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657980/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9deb8078cdb14e2f8df2c38ce46d6396 Command failed (workunit test suites/pjd.sh) on smithi149 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657981] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:20:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657981/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657981/ SELinux denials found on ubuntu@smithi005.front.sepia.ceph.com: ['type=AVC msg=audit(1643899321.854:2660): avc: denied { setgid } for pid=9773 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899355.478:2671): avc: denied { setgid } for pid=10489 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899357.438:2676): avc: denied { setgid } for pid=10510 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899323.741:2665): avc: denied { setgid } for pid=9793 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657984] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:18:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657984/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657984/ SELinux denials found on ubuntu@smithi161.front.sepia.ceph.com: ['type=AVC msg=audit(1643899514.161:2735): avc: denied { setgid } for pid=10722 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899482.359:2718): avc: denied { setgid } for pid=10005 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899512.291:2727): avc: denied { setgid } for pid=10701 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899480.516:2713): avc: denied { setgid } for pid=9985 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657985] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:17:05 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657985/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657985/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c0a462bc6fa649fd940a0f03c79c41dd Command failed (workunit test suites/pjd.sh) on smithi058 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657988] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:15:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657988/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657988/ SELinux denials found on ubuntu@smithi134.front.sepia.ceph.com: ['type=AVC msg=audit(1643899466.409:2656): avc: denied { setgid } for pid=9783 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899468.226:2661): avc: denied { setgid } for pid=9803 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899498.204:2670): avc: denied { setgid } for pid=10507 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899500.032:2672): avc: denied { setgid } for pid=10516 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657989] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657989/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657989/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9d61d87b6204090b9b6e33e7305c8de Command failed (workunit test suites/pjd.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6657993] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:16:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657993/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657993/ SELinux denials found on ubuntu@smithi142.front.sepia.ceph.com: ['type=AVC msg=audit(1643899789.462:2683): avc: denied { setgid } for pid=10556 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899788.102:2679): avc: denied { setgid } for pid=10549 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899755.627:2667): avc: denied { setgid } for pid=9830 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899757.472:2673): avc: denied { setgid } for pid=9850 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899756.975:2671): avc: denied { setgid } for pid=9838 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899789.914:2685): avc: denied { setgid } for pid=10570 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657994] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} ----------------------------------------------------------------- time: 00:10:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657994/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657994/ SELinux denials found on ubuntu@smithi036.front.sepia.ceph.com: ['type=AVC msg=audit(1643899887.276:2673): avc: denied { setgid } for pid=10491 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899851.192:2660): avc: denied { setgid } for pid=9765 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899885.374:2671): avc: denied { setgid } for pid=10482 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899853.481:2665): avc: denied { setgid } for pid=9787 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657995] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 02:25:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657995/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657995/ SELinux denials found on ubuntu@smithi131.front.sepia.ceph.com: ['type=AVC msg=audit(1643899866.007:2677): avc: denied { setgid } for pid=9839 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899864.628:2673): avc: denied { setgid } for pid=9831 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899866.475:2679): avc: denied { setgid } for pid=9851 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899898.842:2687): avc: denied { setgid } for pid=10561 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643899897.006:2685): avc: denied { setgid } for pid=10552 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657997] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:17:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657997/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657997/ SELinux denials found on ubuntu@smithi027.front.sepia.ceph.com: ['type=AVC msg=audit(1643900093.836:2668): avc: denied { setgid } for pid=9809 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900091.887:2662): avc: denied { setgid } for pid=9789 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900093.332:2666): avc: denied { setgid } for pid=9797 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900127.451:2676): avc: denied { setgid } for pid=10526 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900125.514:2674): avc: denied { setgid } for pid=10517 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6657998] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 01:31:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657998/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657998/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d8319cfdce954b3aabf179b38dabfb5c Command failed (workunit test fs/misc/rstats.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/rstats.sh' [6657999] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} ----------------------------------------------------------------- time: 00:24:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657999/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657999/ "2022-02-03T15:05:55.342078+0000 mon.a (mon.0) 1493 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log [6658002] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} ----------------------------------------------------------------- time: 01:54:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658002/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658002/ SELinux denials found on ubuntu@smithi140.front.sepia.ceph.com: ['type=AVC msg=audit(1643900353.595:2669): avc: denied { setgid } for pid=9842 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900351.799:2667): avc: denied { setgid } for pid=9834 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900383.894:2678): avc: denied { setgid } for pid=10551 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900385.705:2680): avc: denied { setgid } for pid=10560 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6658003] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:17:57 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658003/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658003/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f3556ecb791f4299bea4bfa903f5e22b Command failed (workunit test suites/pjd.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6658004] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:17:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658004/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658004/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38d66fe9ab0a4f6f8ae36c8563880a42 reached maximum tries (90) after waiting for 540 seconds [6658005] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:19:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658005/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658005/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c73999dec81f4ca791186d55c1b27402 Command failed (workunit test suites/ffsb.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [6658006] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:24:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658006/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658006/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ca5712ca50704eeb9ca10d3126ed1cd9 timeout expired in wait_until_healthy [6658008] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} ----------------------------------------------------------------- time: 00:16:10 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658008/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658008/ SELinux denials found on ubuntu@smithi064.front.sepia.ceph.com: ['type=AVC msg=audit(1643900566.917:2673): avc: denied { setgid } for pid=10529 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900534.010:2658): avc: denied { setgid } for pid=9792 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900532.171:2656): avc: denied { setgid } for pid=9784 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900566.396:2671): avc: denied { setgid } for pid=10515 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900564.981:2667): avc: denied { setgid } for pid=10508 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6658009] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:25:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658009/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658009/ sentry: https://sentry.ceph.com/organizations/ceph/?query=024faf672b25415c96869af7b366cb91 reached maximum tries (90) after waiting for 540 seconds [6658010] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:16:14 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658010/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658010/ SELinux denials found on ubuntu@smithi065.front.sepia.ceph.com: ['type=AVC msg=audit(1643900904.047:2685): avc: denied { setgid } for pid=10544 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900902.609:2681): avc: denied { setgid } for pid=10535 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900869.448:2667): avc: denied { setgid } for pid=9811 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:groupadd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900904.580:2687): avc: denied { setgid } for pid=10556 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1', 'type=AVC msg=audit(1643900871.267:2675): avc: denied { setgid } for pid=9831 comm="sss_cache" capability=6 scontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:useradd_t:s0-s0:c0.c1023 tclass=capability permissive=1'] [6658013] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658013/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658013/ sentry: https://sentry.ceph.com/organizations/ceph/?query=166a33e67b7d4477a688355f721f326a Command failed on smithi111 with status 1: 'sudo yum upgrade -y linux- firmware' [6658014] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658014/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658014/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cd0c7755c42c4c01bd65a85452ce39fd Command failed on smithi066 with status 1: 'sudo yum upgrade -y linux- firmware' [6658015] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658015/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658015/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f7fdf3d52b7740debf2fd8baa3c5009a Command failed (workunit test suites/pjd.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6658016] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658016/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658016/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c14e34c2a7264815b77f4816a85f7c26 Command failed on smithi105 with status 1: 'sudo yum upgrade -y linux- firmware' [6658018] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658018/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658018/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40473445fb574dc383efdc04e4bb5c26 Command failed on smithi178 with status 1: 'sudo yum upgrade -y linux- firmware' [6658019] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} ----------------------------------------------------------------- time: 00:12:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658019/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658019/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bdf14315f874755b618a3d1ec9d8d5e Test failure: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) [6658021] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658021/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658021/ sentry: https://sentry.ceph.com/organizations/ceph/?query=76112665e4d641979dc3c2dee73b1268 Command failed on smithi037 with status 1: 'sudo yum upgrade -y linux- firmware' [6658022] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658022/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658022/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a9287fbe6c504c7f849ceda2eb80774a Command failed on smithi070 with status 1: 'sudo yum upgrade -y linux- firmware' [6658025] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658025/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=092515cf32504f23842871383febe944 Command failed on smithi065 with status 1: 'sudo yum upgrade -y linux- firmware' [6658030] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658030/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658030/ sentry: https://sentry.ceph.com/organizations/ceph/?query=db761885cf0f491e9fcfec413c846942 Command failed on smithi188 with status 1: 'sudo yum upgrade -y linux- firmware' [6658031] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658031/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658031/ sentry: https://sentry.ceph.com/organizations/ceph/?query=25bc2589c6c04c46bdf83e36491de558 Command failed on smithi093 with status 1: 'sudo yum upgrade -y linux- firmware' [6658033] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658033/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658033/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28fb5f844b314fac8c87bdc68b9d90f0 Command failed on smithi072 with status 1: 'sudo yum upgrade -y linux- firmware' [6658036] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658036/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658036/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df8fb4b427a1485cb26c966a60c1a9c7 Command failed on smithi148 with status 1: 'sudo yum upgrade -y linux- firmware' [6658037] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658037/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658037/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf636755beac4dff876b7ce3f95d578f Command failed on smithi041 with status 1: 'sudo yum upgrade -y linux- firmware' [6658040] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:33 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658040/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658040/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d7c4c64b50a0486b9eb51a653042aa9a Command failed (workunit test suites/pjd.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6658041] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658041/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658041/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fddebc59e3d84915a45f4366de2927bc Command failed on smithi169 with status 1: 'sudo yum upgrade -y linux- firmware' [6658044] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658044/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658044/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f0e7783bb5cb4e1ea93ccc6beddf4a6c Command failed on smithi078 with status 1: 'sudo yum upgrade -y linux- firmware' [6658048] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658048/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658048/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12820f01274144799ec8101c61289fd0 Command failed on smithi151 with status 1: 'sudo yum upgrade -y linux- firmware' [6658050] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658050/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658050/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea5a257bf6174888aa41e2d9fbb5efd6 Command failed on smithi049 with status 1: 'sudo yum upgrade -y linux- firmware' [6658051] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658051/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658051/ sentry: https://sentry.ceph.com/organizations/ceph/?query=41697e03fa9847bdb30dafb60b201b47 Command failed on smithi106 with status 1: 'sudo yum upgrade -y linux- firmware' [6658055] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} ----------------------------------------------------------------- time: 00:13:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658055/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658055/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7f79ebe31e17481db83b1463da37693c Command failed (workunit test suites/pjd.sh) on smithi124 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09fc2e9e470435f1e6dbf462a51853391f4a4cc0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' Dead ================================================================= [6657712] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 00:10:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657712/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657712/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8048962411d047a8bb5890f6adc98dda Failure object was: {'smithi150.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:14:37.658919', 'end': '2022-02-03 11:23:22.058401', 'delta': '0:08:44.399482', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi092.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:14:37.758410', 'end': '2022-02-03 11:23:22.639347', 'delta': '0:08:44.880937', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi029.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:14:42.515522', 'end': '2022-02-03 11:23:26.417113', 'delta': '0:08:43.901591', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657714] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657714/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657714/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ff3ceb1fe62c44b9864030b5ff87713c {'Failure object was': {'smithi106.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:19:59.617699', 'end': '2022-02-03 11:21:00.530085', 'delta': '0:01:00.912386', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657716] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:08:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657716/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657716/ sentry: https://sentry.ceph.com/organizations/ceph/?query=85c6a4d6a8bc438c8f0119bc1533e7b2 {'Failure object was': {'smithi163.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:20:30.013285', 'end': '2022-02-03 11:21:30.858013', 'delta': '0:01:00.844728', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657717] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:06:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657717/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657717/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9595291c7fd7420aafd1359566e17f5a {'Failure object was': {'smithi172.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:15:04.271462', 'end': '2022-02-03 11:16:04.941831', 'delta': '0:01:00.670369', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657718] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} ----------------------------------------------------------------- time: 00:06:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657718/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657718/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9208469b0998409785ebf9b947fc738d {'Failure object was': {'smithi033.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:14:51.821487', 'end': '2022-02-03 11:15:52.507758', 'delta': '0:01:00.686271', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657719] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} ----------------------------------------------------------------- time: 00:06:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657719/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657719/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d809aa7e078d42b0b0bfd948fec72879 {'Failure object was': {'smithi065.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:18:35.834970', 'end': '2022-02-03 11:19:36.579717', 'delta': '0:01:00.744747', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657720] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:10:40 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657720/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657720/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28802e27f4754756b2021c8902601bb0 Failure object was: {'smithi162.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:19:25.347431', 'end': '2022-02-03 11:28:10.202827', 'delta': '0:08:44.855396', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi040.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:19:31.558922', 'end': '2022-02-03 11:28:16.504213', 'delta': '0:08:44.945291', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657722] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:07:58 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657722/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657722/ sentry: https://sentry.ceph.com/organizations/ceph/?query=728d65f102c6455a897b61c93322d749 {'Failure object was': {'smithi071.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:24:51.221494', 'end': '2022-02-03 11:25:52.035088', 'delta': '0:01:00.813594', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657723] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:10:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657723/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657723/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c22c57ca012943b5a0316b64f4e7ac75 Failure object was: {'smithi104.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:20:46.490949', 'end': '2022-02-03 11:29:30.408033', 'delta': '0:08:43.917084', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi039.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:20:49.211087', 'end': '2022-02-03 11:29:32.091636', 'delta': '0:08:42.880549', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657724] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657724/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657724/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b27b32aa31f34cd49cd6157967998f10 Failure object was: {'smithi187.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:19:29.147277', 'end': '2022-02-03 11:28:11.350306', 'delta': '0:08:42.203029', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi170.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:19:29.429149', 'end': '2022-02-03 11:28:12.676029', 'delta': '0:08:43.246880', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi140.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:19:33.602338', 'end': '2022-02-03 11:28:18.515612', 'delta': '0:08:44.913274', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657725] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:06:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657725/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657725/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c27fe66d2ba845d08410d045b574aa4d {'Failure object was': {'smithi152.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:20:29.021475', 'end': '2022-02-03 11:21:29.692603', 'delta': '0:01:00.671128', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657726] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:35 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657726/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657726/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b37f7976b3be4357aa13e5cee8ee00f3 Failure object was: {'smithi107.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:22:50.690726', 'end': '2022-02-03 11:31:34.936245', 'delta': '0:08:44.245519', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi079.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:22:54.562102', 'end': '2022-02-03 11:31:38.621086', 'delta': '0:08:44.058984', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657727] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} ----------------------------------------------------------------- time: 00:10:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657727/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657727/ sentry: https://sentry.ceph.com/organizations/ceph/?query=122c35d6b4ea49e3906574e3e5c710a9 Failure object was: {'smithi059.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:25:03.700751', 'end': '2022-02-03 11:33:45.994964', 'delta': '0:08:42.294213', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi119.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:25:01.925105', 'end': '2022-02-03 11:33:47.765539', 'delta': '0:08:45.840434', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657730] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:08:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657730/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657730/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fb8999f2ea1452ca872c9625b310ae8 {'Failure object was': {'smithi018.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:29:55.079712', 'end': '2022-02-03 11:30:55.972399', 'delta': '0:01:00.892687', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657731] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:06:21 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657731/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657731/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b9f15f35b227407a8d91e0fa72871c38 {'Failure object was': {'smithi026.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:24:43.304733', 'end': '2022-02-03 11:25:44.005544', 'delta': '0:01:00.700811', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657732] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657732/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657732/ sentry: https://sentry.ceph.com/organizations/ceph/?query=def385795b764eb29fa8142988932764 {'Failure object was': {'smithi057.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:31:40.892847', 'end': '2022-02-03 11:32:41.806368', 'delta': '0:01:00.913521', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657733] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:06:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657733/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657733/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a53eef2c38d94c70a45c94e299cf33e0 {'Failure object was': {'smithi060.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:26:19.247081', 'end': '2022-02-03 11:27:19.971290', 'delta': '0:01:00.724209', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657734] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:10:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657734/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657734/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cdace70e61164d2bb853999dc31acefc Failure object was: {'smithi122.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:26:28.366742', 'end': '2022-02-03 11:35:12.070968', 'delta': '0:08:43.704226', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi001.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:26:35.900147', 'end': '2022-02-03 11:35:20.039846', 'delta': '0:08:44.139699', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657737] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} ----------------------------------------------------------------- time: 00:06:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657737/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657737/ sentry: https://sentry.ceph.com/organizations/ceph/?query=34b7ac997a57492b9617bd4436708eee {'Failure object was': {'smithi049.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:29:15.583312', 'end': '2022-02-03 11:30:16.295077', 'delta': '0:01:00.711765', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657739] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:06:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657739/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657739/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d5d45786c0dc47358269be107f943166 {'Failure object was': {'smithi196.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:29:00.703580', 'end': '2022-02-03 11:30:01.417521', 'delta': '0:01:00.713941', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657740] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:10:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657740/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657740/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2aade9c0cd4640168f4e88186d3e6c67 Failure object was: {'smithi204.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:29:56.388891', 'end': '2022-02-03 11:38:39.234358', 'delta': '0:08:42.845467', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi197.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:30:00.933859', 'end': '2022-02-03 11:38:44.025283', 'delta': '0:08:43.091424', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657743] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:10:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657743/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657743/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e26bd76f21d647978b505d6cff91df14 Failure object was: {'smithi103.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:32:43.028809', 'end': '2022-02-03 11:41:28.011428', 'delta': '0:08:44.982619', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi184.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:32:43.151818', 'end': '2022-02-03 11:41:28.988913', 'delta': '0:08:45.837095', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657744] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} ----------------------------------------------------------------- time: 00:10:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657744/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657744/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8a6738dc1dc4af1b1ff2330beca6aba Failure object was: {'smithi167.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:31:38.539810', 'end': '2022-02-03 11:40:22.592148', 'delta': '0:08:44.052338', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi065.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:31:40.052969', 'end': '2022-02-03 11:40:25.250308', 'delta': '0:08:45.197339', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi008.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:31:44.889460', 'end': '2022-02-03 11:40:26.889790', 'delta': '0:08:42.000330', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657746] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:10:40 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657746/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657746/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bdf99c97a64d4722aef2dc0b99e15610 Failure object was: {'smithi148.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:05.023053', 'end': '2022-02-03 11:41:49.863189', 'delta': '0:08:44.840136', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi153.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:05.161656', 'end': '2022-02-03 11:41:51.167671', 'delta': '0:08:46.006015', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657747] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} ----------------------------------------------------------------- time: 00:10:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657747/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657747/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cef209de561f457dabfdb4a253b0b4eb Failure object was: {'smithi173.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:40.009125', 'end': '2022-02-03 11:42:23.180921', 'delta': '0:08:43.171796', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi134.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:39.607898', 'end': '2022-02-03 11:42:23.428748', 'delta': '0:08:43.820850', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657749] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:06:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657749/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657749/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f615b01c50b64e0ca6364c189bcc2860 {'Failure object was': {'smithi066.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:34:27.420975', 'end': '2022-02-03 11:35:28.135200', 'delta': '0:01:00.714225', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657750] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657750/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657750/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ba47179d7fb04a8a8138f00355fd644f Failure object was: {'smithi137.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:11.323100', 'end': '2022-02-03 11:41:56.435457', 'delta': '0:08:45.112357', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi005.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:33:18.116903', 'end': '2022-02-03 11:42:03.030919', 'delta': '0:08:44.914016', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657751] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:10:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657751/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657751/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e689275222cb4821b2d60ee0b60ece8a Failure object was: {'smithi152.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:34:31.606347', 'end': '2022-02-03 11:43:14.578584', 'delta': '0:08:42.972237', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi188.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:34:30.590061', 'end': '2022-02-03 11:43:14.898761', 'delta': '0:08:44.308700', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi096.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:34:30.516903', 'end': '2022-02-03 11:43:15.433560', 'delta': '0:08:44.916657', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657754] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:08:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657754/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657754/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3da4120bbfd34fcfac2512d32ce46f4c {'Failure object was': {'smithi045.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:41:23.553710', 'end': '2022-02-03 11:42:24.421028', 'delta': '0:01:00.867318', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657755] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:06:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657755/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657755/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1bee501ec964b30aa2bd8b67207dcb0 {'Failure object was': {'smithi151.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:34:26.728992', 'end': '2022-02-03 11:35:27.427054', 'delta': '0:01:00.698062', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657756] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} ----------------------------------------------------------------- time: 00:06:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657756/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657756/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b6a985f02d664db69b8a162465feea32 {'Failure object was': {'smithi029.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:36:22.266210', 'end': '2022-02-03 11:37:23.011250', 'delta': '0:01:00.745040', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657757] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:10:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657757/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657757/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4484d7c8a4b4421097f057650667d173 Failure object was: {'smithi067.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:37:05.605230', 'end': '2022-02-03 11:45:49.316452', 'delta': '0:08:43.711222', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi064.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:37:08.512623', 'end': '2022-02-03 11:45:51.515831', 'delta': '0:08:43.003208', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657758] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} ----------------------------------------------------------------- time: 00:06:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657758/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657758/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8fd70ee911f545b4916270b043024d0c {'Failure object was': {'smithi112.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:36:39.287652', 'end': '2022-02-03 11:37:39.942977', 'delta': '0:01:00.655325', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657760] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:07:57 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657760/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657760/ sentry: https://sentry.ceph.com/organizations/ceph/?query=961b1ad4150b43f2be9832b0d53ea3b9 {'Failure object was': {'smithi071.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:42:52.410043', 'end': '2022-02-03 11:43:53.285392', 'delta': '0:01:00.875349', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657761] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657761/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657761/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f568659ec7684db9b3e5db835b40cad0 Failure object was: {'smithi100.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:37:56.848078', 'end': '2022-02-03 11:46:41.589607', 'delta': '0:08:44.741529', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi110.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:37:56.798324', 'end': '2022-02-03 11:46:42.590963', 'delta': '0:08:45.792639', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi026.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:38:04.367684', 'end': '2022-02-03 11:46:48.571275', 'delta': '0:08:44.203591', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657762] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:06:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657762/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657762/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eb421835d96443caad11fb02b6ad5520 {'Failure object was': {'smithi169.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:38:40.030240', 'end': '2022-02-03 11:39:40.720596', 'delta': '0:01:00.690356', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657763] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:10:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657763/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657763/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fce19d2b9d1246b18322917b2e35ea15 Failure object was: {'smithi136.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:39:10.509475', 'end': '2022-02-03 11:47:54.425070', 'delta': '0:08:43.915595', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi062.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:39:16.788862', 'end': '2022-02-03 11:48:00.826007', 'delta': '0:08:44.037145', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657764] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657764/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657764/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6499148334314fd180f34cd6290581ec Failure object was: {'smithi043.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:38:59.543218', 'end': '2022-02-03 11:47:41.755613', 'delta': '0:08:42.212395', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi077.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:38:57.367179', 'end': '2022-02-03 11:47:42.268147', 'delta': '0:08:44.900968', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657767] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:07:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657767/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657767/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0de9022a1bf548248babfbefc3aaf23e {'Failure object was': {'smithi141.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:45:05.765043', 'end': '2022-02-03 11:46:06.680434', 'delta': '0:01:00.915391', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657768] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:06:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657768/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657768/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee7697fdb9904c4a85b7ca4059cbb3fb {'Failure object was': {'smithi170.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]", 'rc': 1, 'start': '2022-02-03 11:40:12.146248', 'end': '2022-02-03 11:41:12.815741', 'delta': '0:01:00.669493', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657769] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657769/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657769/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40969f7460da4f21a55ef50d476ddfd7 Failure object was: {'smithi039.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nStatus code: 504 for https://copr.fedorainfracloud.org/coprs/c eph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)', 'rc': 1, 'start': '2022-02-03 11:41:49.617188', 'end': '2022-02-03 11:48:12.935536', 'delta': '0:06:23.318348', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Status code: 504 for https://copr.fedorainfracloud.org/coprs /ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)'], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi104.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:41:44.615221', 'end': '2022-02-03 11:50:27.475984', 'delta': '0:08:42.860763', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657770] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:09:30 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657770/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657770/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2822f23fd90649eeab0aeef6124efb57 Failure object was: {'smithi166.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:42:49.670755', 'end': '2022-02-03 11:50:22.472880', 'delta': '0:07:32.802125', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi187.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:42:49.990498', 'end': '2022-02-03 11:50:25.657990', 'delta': '0:07:35.667492', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh')Failure object was: {'smithi196.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\nEnabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn\'t connect to server for https://copr.f edorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x 86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]', 'rc': 1, 'start': '2022-02-03 11:42:49.814993', 'end': '2022-02-03 11:50:26.280560', 'delta': '0:07:36.465567', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel- testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Connection timed out]"], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence('tag:yaml.org,2002:seq', data) File "/home/teuthwo rker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663 e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/te uthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3 e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthwor ker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e 0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/h ome/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d00788 2c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'ceph/python3-asyncssh') [6657777] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:01:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657777/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657777/ sentry: https://sentry.ceph.com/organizations/ceph/?query=81e29c0219e24b82aaee2c063d289b76 {'Failure object was': {'smithi066.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\\nEnabling a Copr repository. Please note that this repository is not part\\nof the main distribution, and quality may vary.\\n\\nThe Fedora Project does not exercise any power over the contents of\\nthis repository beyond the rules outlined in the Copr FAQ at\\n,\\nand packages are not held to any quality or security level.\\n\\nPlease do not file bug reports about these packages in Fedora\\nBugzilla. In case of problems, contact the owner of this repository.\\nStatus code: 500 for https://copr.fedorainfracloud.org/coprs/ ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)', 'rc': 1, 'start': '2022-02-03 11:49:45.802392', 'end': '2022-02-03 11:49:46.349065', 'delta': '0:00:00.546673', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Status code: 500 for https://copr.fedorainfracloud.org/coprs /ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)'], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657779] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:01:57 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657779/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657779/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2421d2388f0f44b0b36c60c2484ee709 {'Failure object was': {'smithi029.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': '', 'stderr': 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist\\nInvalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist\\nEnabling a Copr repository. Please note that this repository is not part\\nof the main distribution, and quality may vary.\\n\\nThe Fedora Project does not exercise any power over the contents of\\nthis repository beyond the rules outlined in the Copr FAQ at\\n,\\nand packages are not held to any quality or security level.\\n\\nPlease do not file bug reports about these packages in Fedora\\nBugzilla. In case of problems, contact the owner of this repository.\\nStatus code: 500 for https://copr.fedorainfracloud.org/coprs/ ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)', 'rc': 1, 'start': '2022-02-03 11:49:54.891915', 'end': '2022-02-03 11:49:55.429635', 'delta': '0:00:00.537720', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': [], 'stderr_lines': ['Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Invalid configuration value: failovermethod=priority in /etc/yum.repos.d/epel-testing.repo; Configuration: OptionBinding with id "failovermethod" does not exist', 'Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Status code: 500 for https://copr.fedorainfracloud.org/coprs /ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)'], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657780] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} ----------------------------------------------------------------- time: 00:05:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657780/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657780/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc65b0f61c6c435f82581157c7c09210 {'Failure object was': {'smithi112.front.sepia.ceph.com': {'results': [{'msg': 'non-zero return code', 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'stdout': 'Updating Subscription Management repositories.', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\\nof the main distribution, and quality may vary.\\n\\nThe Fedora Project does not exercise any power over the contents of\\nthis repository beyond the rules outlined in the Copr FAQ at\\n,\\nand packages are not held to any quality or security level.\\n\\nPlease do not file bug reports about these packages in Fedora\\nBugzilla. In case of problems, contact the owner of this repository.\\nStatus code: 500 for https://copr.fedorainfracloud.org/coprs/ ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)', 'rc': 1, 'start': '2022-02-03 11:49:43.261029', 'end': '2022-02-03 11:49:44.100189', 'delta': '0:00:00.839160', 'changed': True, 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': 'None', 'chdir': 'None', 'executable': 'None', 'creates': 'None', 'removes': 'None', 'stdin': 'None'}}, 'stdout_lines': ['Updating Subscription Management repositories.'], 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Status code: 500 for https://copr.fedorainfracloud.org/coprs /ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64 (IP: 3.225.109.36)'], '_ansible_no_log': False, 'item': 'ceph/python3-asyncssh', 'ansible_loop_var': 'item', '_ansible_item_label': 'ceph/python3-asyncssh'}], 'changed': True, 'msg': 'All items completed', 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'ceph/python3-asyncssh')"} [6657836] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:02:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657836/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657836/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2189e9abbc68422ca36514e94b157e0c Failure object was: {'smithi043.front.sepia.ceph.com': {'cache_updated': False, 'stdout': '', 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6302 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open- iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'autoclean': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng- tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb- release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup- bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git- core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml- twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1643891163, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6302 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stdout_lines': [], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6302 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e16 63e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a6 4a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git _teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3 .6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.c eph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/ lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time') [6657861] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657861/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657861/ hit max job timeout [6658012] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:05:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658012/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658012/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6bb5c0794fed4c378b1dd3fea37b8a8d {'Failure object was': {'smithi062.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658020] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:05:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658020/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658020/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d0e1403d94ab479b99c0d61c84e68f15 {'Failure object was': {'smithi074.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658027] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:07:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658027/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658027/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f612c59b366943c0940cc6557abaab02 {'Failure object was': {'smithi031.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658028] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:05:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658028/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658028/ sentry: https://sentry.ceph.com/organizations/ceph/?query=84aea06ff7454aeb807234bb95239893 {'Failure object was': {'smithi018.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658029] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} ----------------------------------------------------------------- time: 00:05:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658029/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658029/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b28222c62ac44768aa39c449ce5e30d {'Failure object was': {'smithi043.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658035] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:05:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658035/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658035/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cb1a4648696743c290cca04fe2375720 {'Failure object was': {'smithi190.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658039] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} ----------------------------------------------------------------- time: 00:05:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658039/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658039/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f1fd4d084da74b2ab32785120d288036 {'Failure object was': {'smithi029.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658042] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:05:39 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658042/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658042/ sentry: https://sentry.ceph.com/organizations/ceph/?query=92e31d6dfdc74525b3938cccf748f092 {'Failure object was': {'smithi055.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658046] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} ----------------------------------------------------------------- time: 00:05:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658046/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658046/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a914574064654577a4718b3f8757d110 {'Failure object was': {'smithi109.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658049] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:05:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658049/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658049/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6551c25924124659abcd6b7735dd3663 {'Failure object was': {'smithi181.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} [6658054] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:05:33 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658054/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658054/ sentry: https://sentry.ceph.com/organizations/ceph/?query=00ff895f75b1493b8771a6ce074bbdc8 {'Failure object was': {'smithi118.front.sepia.ceph.com': {'msg': "Failed to download metadata for repo 'apache-arrow': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['gpg'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64 a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'invocation')"} Pass ================================================================= [6657621] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:22:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657621/ [6657624] fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full} time: 00:20:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657624/ [6657625] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:23:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657625/ [6657627] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 02:13:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657627/ [6657629] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:32:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657629/ [6657631] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:24:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657631/ [6657632] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657632/ [6657633] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:39:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657633/ [6657634] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:45:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657634/ [6657636] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{ubuntu_latest} tasks/fstop} time: 00:10:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657636/ [6657637] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:22:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657637/ [6657640] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:52:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657640/ [6657642] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:16:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657642/ [6657646] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:33 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657646/ [6657647] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657647/ [6657648] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:17:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657648/ [6657650] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:36:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657650/ [6657652] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:10:58 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657652/ [6657654] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:22:48 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657654/ [6657655] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:26:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657655/ [6657657] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:23:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657657/ [6657659] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:08 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657659/ [6657660] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:22:29 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657660/ [6657661] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:30:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657661/ [6657664] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:23:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657664/ [6657666] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:17:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657666/ [6657667] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657667/ [6657669] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:17:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657669/ [6657671] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:25:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657671/ [6657674] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:27:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657674/ [6657675] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:19:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657675/ [6657676] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:20:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657676/ [6657678] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:14:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657678/ [6657679] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:21:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657679/ [6657681] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:31:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657681/ [6657682] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:14:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657682/ [6657684] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:21:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657684/ [6657686] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:18:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657686/ [6657687] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:20:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657687/ [6657689] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:19:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657689/ [6657691] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657691/ [6657694] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:11:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657694/ [6657695] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:22:46 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657695/ [6657696] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:16:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657696/ [6657697] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:02:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657697/ [6657701] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:16:33 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657701/ [6657703] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} time: 00:17:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657703/ [6657705] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:11:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657705/ [6657707] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} time: 00:20:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657707/ [6657708] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:13:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657708/ [6657709] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:03:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657709/ [6657711] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:23:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657711/ [6657713] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:14:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657713/ [6657715] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:56:10 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657715/ [6657721] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:51:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657721/ [6657728] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:16:07 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657728/ [6657729] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 00:53:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657729/ [6657736] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:25:26 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657736/ [6657738] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657738/ [6657741] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:17:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657741/ [6657742] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb} time: 00:11:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657742/ [6657745] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:25:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657745/ [6657748] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:22:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657748/ [6657752] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:13:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657752/ [6657753] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:15:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657753/ [6657759] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:19:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657759/ [6657765] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:24:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657765/ [6657766] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:14:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657766/ [6657771] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:14:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657771/ [6657772] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:17:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657772/ [6657773] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:11:54 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657773/ [6657774] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:32:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657774/ [6657775] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:15:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657775/ [6657776] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:22:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657776/ [6657784] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:34:08 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657784/ [6657785] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657785/ [6657786] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:14:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657786/ [6657787] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:17:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657787/ [6657791] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:55:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657791/ [6657794] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:04:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657794/ [6657796] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:25:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657796/ [6657798] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:20:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657798/ [6657799] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:15:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657799/ [6657801] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 00:59:56 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657801/ [6657802] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:16:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657802/ [6657803] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:20:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657803/ [6657804] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:49:07 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657804/ [6657808] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:27:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657808/ [6657810] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657810/ [6657812] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:17:40 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657812/ [6657818] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:28:14 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657818/ [6657819] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:25:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657819/ [6657821] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:15:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657821/ [6657822] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:21:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657822/ [6657824] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 03:02:53 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657824/ [6657826] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 03:18:58 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657826/ [6657827] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:22:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657827/ [6657829] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:21:42 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657829/ [6657830] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657830/ [6657831] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:33:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657831/ [6657832] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:18:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657832/ [6657834] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:23:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657834/ [6657837] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:17:48 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657837/ [6657838] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:57:30 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657838/ [6657839] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:22:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657839/ [6657842] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:16:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657842/ [6657848] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:11:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657848/ [6657851] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:20:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657851/ [6657852] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/lockdep} time: 00:18:55 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657852/ [6657853] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:14:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657853/ [6657854] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:12:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657854/ [6657856] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:32:59 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657856/ [6657858] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657858/ [6657859] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:11:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657859/ [6657864] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:07:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657864/ [6657867] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:27:12 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657867/ [6657868] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:11:26 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657868/ [6657871] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 01:00:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657871/ [6657873] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:16:56 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657873/ [6657874] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 00:59:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657874/ [6657876] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:23:18 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657876/ [6657877] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:11:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657877/ [6657878] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:49:20 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657878/ [6657880] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:20:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657880/ [6657881] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:22:08 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657881/ [6657884] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:18:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657884/ [6657889] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:07:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657889/ [6657890] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:28:11 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657890/ [6657891] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:33:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657891/ [6657894] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:18:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657894/ [6657896] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:37 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657896/ [6657900] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:26:10 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657900/ [6657901] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:25:08 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657901/ [6657903] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:13:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657903/ [6657906] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:18:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657906/ [6657909] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:50:36 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657909/ [6657910] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:15:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657910/ [6657912] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:27:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657912/ [6657915] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:16:43 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657915/ [6657919] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:12:32 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657919/ [6657923] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/valgrind} time: 00:56:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657923/ [6657925] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:29:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657925/ [6657926] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:57:17 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657926/ [6657930] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:22 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657930/ [6657931] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:21:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657931/ [6657934] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:13:26 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657934/ [6657935] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:09:19 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657935/ [6657936] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:25:52 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657936/ [6657937] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:15:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657937/ [6657938] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:19:58 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657938/ [6657939] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:16:13 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657939/ [6657940] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 00:53:48 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657940/ [6657944] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:44:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657944/ [6657945] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:04:02 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657945/ [6657946] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:25:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657946/ [6657947] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:21:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657947/ [6657948] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:06:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657948/ [6657952] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657952/ [6657959] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:15:25 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657959/ [6657961] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:21:00 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657961/ [6657962] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:08:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657962/ [6657963] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:29:16 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657963/ [6657965] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:16:08 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657965/ [6657968] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:16:23 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657968/ [6657970] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:19:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657970/ [6657971] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:18:51 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657971/ [6657973] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:03:15 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657973/ [6657975] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:26 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657975/ [6657976] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:58:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657976/ [6657977] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:22:07 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657977/ [6657978] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:34 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657978/ [6657982] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:22:09 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657982/ [6657983] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:06:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657983/ [6657986] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:16:07 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657986/ [6657987] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:17:49 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657987/ [6657990] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:54:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657990/ [6657991] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:15:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657991/ [6657992] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:24:40 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657992/ [6657996] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:55:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6657996/ [6658000] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:50 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658000/ [6658001] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:22:47 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658001/ [6658007] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:35:28 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658007/ [6658011] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:22:01 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658011/ [6658017] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:18:31 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658017/ [6658023] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:12:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658023/ [6658024] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:39:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658024/ [6658026] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:50:21 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658026/ [6658032] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:12:27 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658032/ [6658034] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:24 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658034/ [6658038] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:13:06 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658038/ [6658043] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:23:38 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658043/ [6658045] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:44 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658045/ [6658047] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:16:45 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658047/ [6658052] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:00:04 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658052/ [6658053] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:23:49 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658053/ [6658056] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:21:05 info: http://pulpito.ceph.com/vshankar-2022-02-03_04:16:32-fs-wip-vshankar-testing-20220201-113815-testing-default-smithi/6658056/