2020-10-03T00:52:57.763 INFO:root:teuthology version: 1.0.0-75888fc 2020-10-03T00:52:57.765 INFO:teuthology.results:seed: '6396' 2020-10-03T01:37:08.565 INFO:root:teuthology version: 1.0.0-75888fc 2020-10-03T01:37:08.586 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-10-03T11:18:21.020 INFO:teuthology.results:Tests finished! gathering results... 2020-10-03T11:18:32.825 INFO:scrape:Found 100 jobs 2020-10-03T11:18:36.972 INFO:scrape:Found 5 distinct failure reasons 2020-10-03T11:18:36.973 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/concurrent.sh 2020-10-03T11:18:36.973 INFO:scrape:1 jobs: ['5470511'] 2020-10-03T11:18:36.973 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2020-10-03T11:18:36.974 INFO:scrape: 2020-10-03T11:18:36.974 INFO:scrape:Failure: "2020-10-03T01:28:55.351615+0000 mon.a (mon.0) 124 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 31 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log 2020-10-03T11:18:36.975 INFO:scrape:4 jobs: ['5470460', '5470481', '5470457', '5470440'] 2020-10-03T11:18:36.975 INFO:scrape:suites intersection: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap'] 2020-10-03T11:18:36.975 INFO:scrape:suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/few', 'msgr-failures/many', 'tasks/krbd_data_pool}', 'tasks/krbd_huge_osdmap}', 'tasks/krbd_udev_symlinks}', 'tasks/rbd_concurrent}'] 2020-10-03T11:18:36.976 INFO:scrape: 2020-10-03T11:18:36.976 INFO:scrape:Failure: Command failed on smithi066 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeatkydbgp -r -- '-g auto -g blockdev -x clone'" 2020-10-03T11:18:36.976 INFO:scrape:2 jobs: ['5470431', '5470482'] 2020-10-03T11:18:36.977 INFO:scrape:suites intersection: ['conf', 'krbd/singleton/{bluestore-bitmap', 'tasks/rbd_xfstests}'] 2020-10-03T11:18:36.977 INFO:scrape:suites union: ['conf', 'krbd/singleton/{bluestore-bitmap', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_xfstests}'] 2020-10-03T11:18:36.977 INFO:scrape: 2020-10-03T11:18:36.978 INFO:scrape:Failure: Command failed on smithi085 with status 120: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 1 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.1 image_client.1' 2020-10-03T11:18:36.978 INFO:scrape:1 jobs: ['5470451'] 2020-10-03T11:18:36.979 INFO:scrape:suites: ['clusters/3-node', 'conf', 'features/object-map', 'krbd/fsx/{ceph/ceph', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-3-client}'] 2020-10-03T11:18:36.979 INFO:scrape: 2020-10-03T11:18:36.979 INFO:scrape:Failure: "2020-10-03T01:28:55.515926+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4413.0:851737 3.15 3:aadcc618:::rbd_data.113da8603ec4.0000000000000002:head [truncate 4194304] snapc 65=[65] ondisk+write+known_if_redirected e121) initiated 2020-10-03T01:28:24.639339+0000 currently delayed" in cluster log 2020-10-03T11:18:36.980 INFO:scrape:1 jobs: ['5470447'] 2020-10-03T11:18:36.980 INFO:scrape:suites: ['clusters/3-node', 'conf', 'features/no-object-map', 'krbd/fsx/{ceph/ceph', 'objectstore/bluestore-bitmap', 'randomized-striping-on}', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-1-client}'] 2020-10-03T11:18:36.980 INFO:scrape: 2020-10-03T11:18:36.986 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/ failed: 9 dead: 0 running: 0 waiting: 0 queued: 0 passed: 91 Fail ================================================================= [5470431] krbd/singleton/{bluestore-bitmap conf msgr-failures/few tasks/rbd_xfstests} ----------------------------------------------------------------- time: 08:56:41 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470431/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470431/ sentry: https://sentry.ceph.com/sentry/sepia/?query=984a349efe904ac89ca368f11a49b2f4 Command failed on smithi066 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeatkydbgp -r -- '-g auto -g blockdev -x clone'" [5470440] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_huge_osdmap} ----------------------------------------------------------------- time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470440/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470440/ "2020-09-27T16:29:50.628135+0000 mon.a (mon.0) 40 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5470447] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:44:50 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470447/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470447/ "2020-10-03T01:28:55.515926+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4413.0:851737 3.15 3:aadcc618:::rbd_data.113da8603ec4.0000000000000002:head [truncate 4194304] snapc 65=[65] ondisk+write+known_if_redirected e121) initiated 2020-10-03T01:28:24.639339+0000 currently delayed" in cluster log [5470451] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:22:23 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470451/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470451/ sentry: https://sentry.ceph.com/sentry/sepia/?query=5fb5c3ead87a49028fee9b9466ac7259 Command failed on smithi085 with status 120: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 1 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.1 image_client.1' [5470457] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_symlinks} ----------------------------------------------------------------- time: 00:17:33 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470457/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470457/ "2020-10-03T01:32:25.329639+0000 mon.a (mon.0) 34 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5470460] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_concurrent} ----------------------------------------------------------------- time: 00:17:46 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470460/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470460/ "2020-10-03T01:28:55.351615+0000 mon.a (mon.0) 124 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 31 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log [5470481] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_data_pool} ----------------------------------------------------------------- time: 00:32:54 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470481/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470481/ "2020-10-03T01:57:16.037483+0000 mon.b (mon.0) 33 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5470482] krbd/singleton/{bluestore-bitmap conf msgr-failures/many tasks/rbd_xfstests} ----------------------------------------------------------------- time: 09:44:47 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470482/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470482/ sentry: https://sentry.ceph.com/sentry/sepia/?query=277c1a8892554a2f801292d02178ed13 Command failed on smithi135 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludekcjkgd93 -r -- '-g auto -g blockdev -x clone'" [5470511] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 05:00:34 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470511/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470511/ sentry: https://sentry.ceph.com/sentry/sepia/?query=042151a45cd94deca74fad3a4a893696 Command failed (workunit test rbd/concurrent.sh) on smithi033 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d747ed63ed56e4d0f065478d3392b2eeb48ddf64 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/concurrent.sh' Pass ================================================================= [5470427] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_deep_flatten} time: 00:10:54 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470427/ [5470428] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:24:58 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470428/ [5470429] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_fio} time: 00:42:54 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470429/ [5470430] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_data_pool} time: 00:23:04 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470430/ [5470432] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_fio} time: 01:25:27 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470432/ [5470433] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470433/ [5470434] krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_pages_required} time: 00:09:48 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470434/ [5470435] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_exclusive_option} time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470435/ [5470436] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:28:48 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470436/ [5470437] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_kernel_untar_build} time: 00:17:39 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470437/ [5470438] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_fallocate} time: 00:11:56 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470438/ [5470439] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470439/ [5470441] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:51:32 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470441/ [5470442] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:14:36 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470442/ [5470443] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_discard} time: 00:11:24 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470443/ [5470444] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:06 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470444/ [5470445] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:16:05 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470445/ [5470446] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_namespaces} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470446/ [5470448] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:29:08 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470448/ [5470449] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_enumerate} time: 00:30:23 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470449/ [5470450] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470450/ [5470452] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:18 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470452/ [5470453] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_netns} time: 00:12:43 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470453/ [5470454] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_huge_image} time: 00:09:17 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470454/ [5470455] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:26:02 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470455/ [5470456] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_fio} time: 01:20:25 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470456/ [5470458] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:14:42 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470458/ [5470459] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470459/ [5470461] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_huge_tickets} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470461/ [5470462] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:25:10 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470462/ [5470463] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:19:33 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470463/ [5470464] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} time: 00:10:02 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470464/ [5470465] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} time: 00:23:06 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470465/ [5470466] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_image_read} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470466/ [5470467] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_modprobe} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470467/ [5470468] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:28:22 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470468/ [5470469] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 02:24:44 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470469/ [5470470] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_kernel} time: 00:10:25 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470470/ [5470471] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470471/ [5470472] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470472/ [5470473] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:38:51 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470473/ [5470474] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_unmap} time: 00:15:36 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470474/ [5470475] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470475/ [5470476] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:28:26 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470476/ [5470477] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_simple_big} time: 00:39:52 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470477/ [5470478] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_msgr_segments} time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470478/ [5470479] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:30:11 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470479/ [5470480] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_fio} time: 00:43:40 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470480/ [5470483] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:57:00 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470483/ [5470484] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_exclusive_option} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470484/ [5470485] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:33:48 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470485/ [5470486] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:17:34 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470486/ [5470487] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_fallocate} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470487/ [5470488] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470488/ [5470489] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_huge_osdmap} time: 00:12:10 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470489/ [5470490] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:51:15 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470490/ [5470491] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:15:35 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470491/ [5470492] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_parent_overlap} time: 00:09:41 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470492/ [5470493] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:58 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470493/ [5470494] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} time: 01:27:38 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470494/ [5470495] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_namespaces} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470495/ [5470496] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:37:01 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470496/ [5470497] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:13:49 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470497/ [5470498] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} time: 00:09:54 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470498/ [5470499] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} time: 00:24:16 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470499/ [5470500] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_enumerate} time: 00:47:43 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470500/ [5470501] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:17:24 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470501/ [5470502] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:26:45 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470502/ [5470503] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470503/ [5470504] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_netns} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470504/ [5470505] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_read_only} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470505/ [5470506] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470506/ [5470507] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:42:10 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470507/ [5470508] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_symlinks} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470508/ [5470509] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:11:09 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470509/ [5470510] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:41:27 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470510/ [5470512] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_huge_tickets} time: 00:11:06 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470512/ [5470513] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:30:04 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470513/ [5470514] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:20:11 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470514/ [5470515] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_image_read} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470515/ [5470516] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_whole_object_zeroout} time: 00:09:49 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470516/ [5470517] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:23:54 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470517/ [5470518] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_fio} time: 01:21:53 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470518/ [5470519] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_kernel} time: 00:10:43 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470519/ [5470520] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:14:15 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470520/ [5470521] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:10:28 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470521/ [5470522] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:31:34 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470522/ [5470523] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_unmap} time: 00:15:51 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470523/ [5470524] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470524/ [5470525] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:26:23 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470525/ [5470526] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_simple_big} time: 00:39:50 info: http://pulpito.ceph.com/teuthology-2020-09-26_03:20:02-krbd-master-testing-basic-smithi/5470526/