Found 308 jobs Ignoring malparsed backtrace: ceph version 16.0.0-2985-g904d32d8ca5 (904d32d8ca505c5fe33737c2692b0fc2f564b61c) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f23d58fac66] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55e2ee97b124] Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-06-28T08:03:32.310 INFO:teuthology.orchestra.run.smithi095:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json , 2020-06-28T08:03:32.592 INFO:teuthology.orchestra.run.smithi095.stdout: Ignoring malparsed backtrace: 2020-06-28T08:04:34.271 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2020-06-28T08:04:34.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi095.front.sepia.ceph.com , 2020-06-28T08:04:34.515 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2020-06-28T08:05:55.387 INFO:teuthology.orchestra.run.smithi095.stdout: ceph-radosgw x86_64 2:16.0.0-2985.g904d32d8ca5.el8 @ceph 41 M , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout: , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary Ignoring malparsed backtrace: 2020-06-28T08:05:59.993 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. , 2020-06-28T08:05:59.994 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test , 2020-06-28T08:06:00.034 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. Ignoring malparsed backtrace: 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. , 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction , 2020-06-28T08:06:05.310 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 Ignoring malparsed backtrace: 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 @rhel-8-for-x86_64-appstream-rpms 2.5 M , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jsonpatch noarch 1.21-2.el8 @rhel-8-for-x86_64-appstream-rpms 55 k Ignoring malparsed backtrace: 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-html5lib noarch 1:0.999999999-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 6.2-6.el8 @epel 4.0 k , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k Ignoring malparsed backtrace: 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: rbd-mirror-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 17/81 , 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 , 2020-06-28T08:06:14.975 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 Ignoring malparsed backtrace: 2020-06-28T08:06:22.735 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-html5lib-1:0.999999999-6.el8.noarch 59/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 55/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : librgw2-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 56/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.171 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : libquadmath-8.3.1-5.el8.x86_64 16/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 17/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 34/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 35/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 36/81 Ignoring malparsed backtrace: 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-mako-1.0.6-13.el8.noarch , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-markupsafe-0.23-19.el8.x86_64 , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-more-itertools-7.2.0-3.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch , 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 , 2020-06-28T08:06:46.725 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.4.1-12.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:52.284 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================ , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:Remove 1 Package Ignoring malparsed backtrace: No packages marked for removal. , 2020-06-28T08:06:59.438 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. , 2020-06-28T08:06:59.439 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. Ignoring malparsed backtrace: 2020-06-28T08:07:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 , 2020-06-28T08:07:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/1 , 2020-06-28T08:07:02.816 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:08.070 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_6 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:14.537 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout: Package Arch Version Repository Size , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ Ignoring malparsed backtrace: , Problem: conflicting requests , - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 Ignoring malparsed backtrace: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 , 2020-06-28T08:07:25.850 INFO:teuthology.orchestra.run.smithi094.stdout:No match for argument: rbd-fuse , No packages marked for removal. Ignoring malparsed backtrace: 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1451us[+1451us] +/- 65ms , 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv04.front.sepia.ceph.com 2 6 377 15 -1662us[-1662us] +/- 71ms , 2020-06-28T08:07:30.173 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab Ignoring malparsed backtrace: 2020-06-28T08:07:48.947 INFO:teuthology.task.internal:Tidying up after the test... , 2020-06-28T08:07:48.948 INFO:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2020-06-28T08:07:48.951 INFO:teuthology.orchestra.run.smithi095:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest Ignoring malparsed backtrace: ceph version 16.0.0-2985-g904d32d8ca5 (904d32d8ca505c5fe33737c2692b0fc2f564b61c) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f23d58fac66] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55e2ee97b124] Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-06-28T08:03:32.310 INFO:teuthology.orchestra.run.smithi095:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json , 2020-06-28T08:03:32.592 INFO:teuthology.orchestra.run.smithi095.stdout: Ignoring malparsed backtrace: 2020-06-28T08:04:34.271 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2020-06-28T08:04:34.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi095.front.sepia.ceph.com , 2020-06-28T08:04:34.515 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2020-06-28T08:05:55.387 INFO:teuthology.orchestra.run.smithi095.stdout: ceph-radosgw x86_64 2:16.0.0-2985.g904d32d8ca5.el8 @ceph 41 M , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout: , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary Ignoring malparsed backtrace: 2020-06-28T08:05:59.993 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. , 2020-06-28T08:05:59.994 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test , 2020-06-28T08:06:00.034 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. Ignoring malparsed backtrace: 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. , 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction , 2020-06-28T08:06:05.310 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 Ignoring malparsed backtrace: 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 @rhel-8-for-x86_64-appstream-rpms 2.5 M , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jsonpatch noarch 1.21-2.el8 @rhel-8-for-x86_64-appstream-rpms 55 k Ignoring malparsed backtrace: 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-html5lib noarch 1:0.999999999-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 6.2-6.el8 @epel 4.0 k , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k Ignoring malparsed backtrace: 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: rbd-mirror-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 17/81 , 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 , 2020-06-28T08:06:14.975 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 Ignoring malparsed backtrace: 2020-06-28T08:06:22.735 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-html5lib-1:0.999999999-6.el8.noarch 59/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 55/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : librgw2-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 56/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.171 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : libquadmath-8.3.1-5.el8.x86_64 16/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 17/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 34/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 35/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 36/81 Ignoring malparsed backtrace: 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-mako-1.0.6-13.el8.noarch , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-markupsafe-0.23-19.el8.x86_64 , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-more-itertools-7.2.0-3.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch , 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 , 2020-06-28T08:06:46.725 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.4.1-12.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:52.284 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================ , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:Remove 1 Package Ignoring malparsed backtrace: No packages marked for removal. , 2020-06-28T08:06:59.438 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. , 2020-06-28T08:06:59.439 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. Ignoring malparsed backtrace: 2020-06-28T08:07:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 , 2020-06-28T08:07:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/1 , 2020-06-28T08:07:02.816 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:08.070 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_6 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:14.537 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout: Package Arch Version Repository Size , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ Ignoring malparsed backtrace: , Problem: conflicting requests , - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 Ignoring malparsed backtrace: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 , 2020-06-28T08:07:25.850 INFO:teuthology.orchestra.run.smithi094.stdout:No match for argument: rbd-fuse , No packages marked for removal. Ignoring malparsed backtrace: 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1451us[+1451us] +/- 65ms , 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv04.front.sepia.ceph.com 2 6 377 15 -1662us[-1662us] +/- 71ms , 2020-06-28T08:07:30.173 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab Ignoring malparsed backtrace: 2020-06-28T08:07:48.947 INFO:teuthology.task.internal:Tidying up after the test... , 2020-06-28T08:07:48.948 INFO:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2020-06-28T08:07:48.951 INFO:teuthology.orchestra.run.smithi095:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest Ignoring malparsed backtrace: ceph version 16.0.0-2985-g904d32d8ca5 (904d32d8ca505c5fe33737c2692b0fc2f564b61c) pacific (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7f23d58fac66] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x4f4) [0x55e2ee97b124] Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-06-28T08:03:32.310 INFO:teuthology.orchestra.run.smithi095:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json , 2020-06-28T08:03:32.592 INFO:teuthology.orchestra.run.smithi095.stdout: Ignoring malparsed backtrace: 2020-06-28T08:04:34.271 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2020-06-28T08:04:34.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi095.front.sepia.ceph.com , 2020-06-28T08:04:34.515 INFO:teuthology.orchestra.run.smithi095:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2020-06-28T08:05:55.387 INFO:teuthology.orchestra.run.smithi095.stdout: ceph-radosgw x86_64 2:16.0.0-2985.g904d32d8ca5.el8 @ceph 41 M , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout: , 2020-06-28T08:05:55.388 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary Ignoring malparsed backtrace: 2020-06-28T08:05:59.993 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. , 2020-06-28T08:05:59.994 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test , 2020-06-28T08:06:00.034 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. Ignoring malparsed backtrace: 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. , 2020-06-28T08:06:05.226 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction , 2020-06-28T08:06:05.310 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 Ignoring malparsed backtrace: 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 @rhel-8-for-x86_64-appstream-rpms 2.5 M , 2020-06-28T08:06:09.773 INFO:teuthology.orchestra.run.smithi095.stdout: python3-jsonpatch noarch 1.21-2.el8 @rhel-8-for-x86_64-appstream-rpms 55 k Ignoring malparsed backtrace: 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-html5lib noarch 1:0.999999999-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 6.2-6.el8 @epel 4.0 k , 2020-06-28T08:06:10.386 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 2.0-4.el8 @epel 28 k Ignoring malparsed backtrace: 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: rbd-mirror-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 17/81 , 2020-06-28T08:06:14.784 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 , 2020-06-28T08:06:14.975 INFO:teuthology.orchestra.run.smithi095.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 18/81 Ignoring malparsed backtrace: 2020-06-28T08:06:22.735 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : python3-html5lib-1:0.999999999-6.el8.noarch 59/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 55/81 , 2020-06-28T08:06:22.748 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : librgw2-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 56/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.171 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : libquadmath-8.3.1-5.el8.x86_64 16/81 , 2020-06-28T08:06:42.172 INFO:teuthology.orchestra.run.smithi095.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 17/81 Ignoring malparsed backtrace: 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 34/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 35/81 , 2020-06-28T08:06:42.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 36/81 Ignoring malparsed backtrace: 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-mako-1.0.6-13.el8.noarch , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-markupsafe-0.23-19.el8.x86_64 , 2020-06-28T08:06:46.188 INFO:teuthology.orchestra.run.smithi095.stdout: python3-more-itertools-7.2.0-3.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch , 2020-06-28T08:06:46.724 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 , 2020-06-28T08:06:46.725 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.4.1-12.el8.noarch Ignoring malparsed backtrace: 2020-06-28T08:06:52.284 INFO:teuthology.orchestra.run.smithi095.stdout:Transaction Summary , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================ , 2020-06-28T08:06:52.285 INFO:teuthology.orchestra.run.smithi095.stdout:Remove 1 Package Ignoring malparsed backtrace: No packages marked for removal. , 2020-06-28T08:06:59.438 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. , 2020-06-28T08:06:59.439 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. Ignoring malparsed backtrace: 2020-06-28T08:07:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 , 2020-06-28T08:07:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/1 , 2020-06-28T08:07:02.816 INFO:teuthology.orchestra.run.smithi094.stdout: Erasing : ceph-fuse-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:08.070 INFO:teuthology.orchestra.run.smithi095.stdout: Erasing : librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_64 3/3 , 2020-06-28T08:07:08.214 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:16.0.0-2985.g904d32d8ca5.el8.x86_6 1/3 Ignoring malparsed backtrace: 2020-06-28T08:07:14.537 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout: Package Arch Version Repository Size , 2020-06-28T08:07:14.538 INFO:teuthology.orchestra.run.smithi095.stdout:================================================================================================================ Ignoring malparsed backtrace: , Problem: conflicting requests , - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 Ignoring malparsed backtrace: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 , 2020-06-28T08:07:25.850 INFO:teuthology.orchestra.run.smithi094.stdout:No match for argument: rbd-fuse , No packages marked for removal. Ignoring malparsed backtrace: 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1451us[+1451us] +/- 65ms , 2020-06-28T08:07:30.172 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv04.front.sepia.ceph.com 2 6 377 15 -1662us[-1662us] +/- 71ms , 2020-06-28T08:07:30.173 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab Ignoring malparsed backtrace: 2020-06-28T08:07:48.947 INFO:teuthology.task.internal:Tidying up after the test... , 2020-06-28T08:07:48.948 INFO:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest , 2020-06-28T08:07:48.951 INFO:teuthology.orchestra.run.smithi095:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest Found 10 distinct failure reasons Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 1 jobs: ['5185636'] suites: ['msgr-failures/many', 'msgr/async', 'objectstore/bluestore-stupid', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{rhel_8}}'] Failure: Scrubbing terminated -- not all pgs were active and clean. 2 jobs: ['5185781', '5185717'] suites intersection: ['msgr/async-v2only', 'rados'] suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'd-balancer/crush-compat', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/singleton/{all/lost-unfound-delete', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'thrashers/none', 'thrashosds-health', 'workloads/radosbench-high-concurrency}'] Failure: "2020-06-28T07:28:04.267158+0000 mon.a (mon.0) 49 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log 4 jobs: ['5185577', '5185562', '5185740', '5185716'] suites intersection: ['msgr-failures/few', 'rados'] suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/multimon/{clusters/21', 'rados/singleton/{all/random-eio', 'rados/singleton/{all/recovery-preemption', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/mon_clock_no_skews}', 'tasks/mon_recovery', 'validater/lockdep}'] Failure: reached maximum tries (180) after waiting for 180 seconds 5 jobs: ['5185788', '5185722', '5185755', '5185626', '5185655'] suites intersection: ['ceph', 'clusters/{openstack', 'distro$/{centos_7.6}', 'rados', 'thrashosds-health', 'three-plus-one}'] suites union: ['1-install/luminous', '1-install/mimic', '1-install/mimic-v1only', '1-install/nautilus', '1-install/octopus', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{centos_7.6}', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'rados', 'rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'thrashers/careful', 'thrashers/default', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] Dead: b'2020-06-28T13:21:51.442 DEBUG:teuthology.exit:Finished running handlers' 2 jobs: ['5185684', '5185793'] suites intersection: ['fixed-2', 'start}'] suites union: ['fixed-2', 'rados/cephadm/smoke/{distro/centos_8.0', 'rados/cephadm/smoke/{distro/ubuntu_18.04_podman', 'start}'] Failure: Command failed (workunit test osd/osd-recovery-stats.sh) on smithi114 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=904d32d8ca505c5fe33737c2692b0fc2f564b61c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-recovery-stats.sh' 1 jobs: ['5185587'] suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/osd}'] Failure: Command failed (workunit test mon/pg_autoscaler.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=904d32d8ca505c5fe33737c2692b0fc2f564b61c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pg_autoscaler.sh' 1 jobs: ['5185534'] suites: ['msgr-failures/many', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/pg-autoscaler', 'supported-random-distro$/{rhel_8}}'] Failure: "2020-06-28T08:05:32.672060+0000 osd.4 (osd.4) 3 : cluster [WRN] slow request osd_op(client.4513.0:277 3.8 3.381257a8 (undecoded) ondisk+write+known_if_redirected e23) initiated 2020-06-28T08:05:02.275985+0000 currently delayed" in cluster log 2 jobs: ['5185656', '5185783'] suites intersection: ['clusters/{fixed-2', 'openstack}', 'rados', 'rados/basic/{ceph'] suites union: ['clusters/{fixed-2', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'tasks/rados_python}', 'tasks/readwrite}'] Failure: Command failed on smithi152 with status 1: 'cd /home/ubuntu/cephtest/cos && chmod +x *.sh' 2 jobs: ['5185604', '5185591'] suites intersection: ['openstack', 'rados/perf/{ceph', 'settings/optimized', 'ubuntu_latest'] suites union: ['objectstore/bluestore-basic-min-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack', 'rados/perf/{ceph', 'settings/optimized', 'ubuntu_latest', 'workloads/cosbench_64K_read_write}', 'workloads/cosbench_64K_write}'] Failure: wait_for_clean: failed before timeout expired 1 jobs: ['5185685'] suites: ['clusters/{fixed-2', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_8}', 'tasks/repair_test}']