2022-10-08T14:52:41.743 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-08T14:52:41.745 INFO:teuthology.results:subset: '8/10' 2022-10-08T14:52:41.745 INFO:teuthology.results:seed: '7333' 2022-10-08T16:29:17.023 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-08T16:29:17.042 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-10-08T18:55:27.309 INFO:teuthology.results:Tests finished! gathering results... 2022-10-08T18:55:28.743 INFO:scrape:Found 14 jobs 2022-10-08T18:55:28.948 INFO:scrape:Found 3 distinct failure reasons 2022-10-08T18:55:28.949 INFO:scrape:Failure: {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'failures': [], 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph-selinux', 'ceph-common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python-cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': 'Unknown Error occured: ceph-mgr = 2:17.2.3-910.g51455dec.el8 is needed by (installed) ceph-mgr-dashboard-2:17.2.3-910.g51455dec.el8.noarch', 'rc': 1, 'results': []}} 2022-10-08T18:55:28.949 INFO:scrape:1 jobs: ['7058423'] 2022-10-08T18:55:28.950 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd_api', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04'] 2022-10-08T18:55:28.950 INFO:scrape: 2022-10-08T18:55:28.951 INFO:scrape:Failure: {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'msg': "failed to transfer file to /home/teuthworker/.ansible/tmp/ansible-local-677mx3ioinq/tmph_jwa6_z /home/ubuntu/.ansible/tmp/ansible-tmp-1665245014.72627-4578-237334952884069/AnsiballZ_authorized_key.py:\n\ndd: failed to open '/home/ubuntu/.ansible/tmp/ansible-tmp-1665245014.72627-4578-237334952884069/AnsiballZ_authorized_key.py': No such file or directory\n"}} 2022-10-08T18:55:28.951 INFO:scrape:1 jobs: ['7058435'] 2022-10-08T18:55:28.952 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/radosbench', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0'] 2022-10-08T18:55:28.952 INFO:scrape: 2022-10-08T18:55:28.953 INFO:scrape:Failure: {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'msg': 'Failed to connect to the host via ssh: ssh: connect to host smithi036.front.sepia.ceph.com port 22: No route to host'}} 2022-10-08T18:55:28.953 INFO:scrape:1 jobs: ['7058432'] 2022-10-08T18:55:28.954 INFO:scrape:suites: ['0-start', '1-tasks', 'mon_election/connectivity', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:pacific-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'workload/{ec-rados-default'] 2022-10-08T18:55:28.954 INFO:scrape: 2022-10-08T18:55:28.956 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/ failed: 1 dead: 2 running: 0 waiting: 0 queued: 0 passed: 11 Fail ================================================================= [7058435] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} ----------------------------------------------------------------- time: 00:07:34 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058435/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058435/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c8a3639ec4ef443aa608d71ce4b8fddb {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'msg': "failed to transfer file to /home/teuthworker/.ansible/tmp/ansible- local-677mx3ioinq/tmph_jwa6_z /home/ubuntu/.ansible/tmp/ansible-tmp-1665245 014.72627-4578-237334952884069/AnsiballZ_authorized_key.py:\n\ndd: failed to open '/home/ubuntu/.ansible/tmp/ansible- tmp-1665245014.72627-4578-237334952884069/AnsiballZ_authorized_key.py': No such file or directory\n"}} Dead ================================================================= [7058423] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} ----------------------------------------------------------------- time: 00:02:39 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058423/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058423/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0adbe925b8494348ae419ab95107a975 {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'failures': [], 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph- selinux', 'ceph-common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python- cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': 'Unknown Error occured: ceph-mgr = 2:17.2.3-910.g51455dec.el8 is needed by (installed) ceph-mgr-dashboard-2:17.2.3-910.g51455dec.el8.noarch', 'rc': 1, 'results': []}} [7058432] upgrade:pacific-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058432/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058432/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee69d93bd2244c3bad1ca29c4fef03fa {'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False, 'msg': 'Failed to connect to the host via ssh: ssh: connect to host smithi036.front.sepia.ceph.com port 22: No route to host'}} Pass ================================================================= [7058411] upgrade:pacific-x/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:24:28 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058411/ [7058414] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} time: 02:03:26 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058414/ [7058417] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} time: 02:18:20 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058417/ [7058420] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} time: 02:04:22 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058420/ [7058426] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} time: 01:51:38 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058426/ [7058429] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} time: 02:21:46 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058429/ [7058438] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} time: 02:14:44 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058438/ [7058441] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} time: 02:16:26 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058441/ [7058444] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} time: 02:06:43 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058444/ [7058448] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} time: 01:41:14 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058448/ [7058450] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} time: 02:20:53 info: http://pulpito.ceph.com/teuthology-2022-10-08_14:23:02-upgrade:pacific-x-main-distro-default-smithi/7058450/