2021-11-11T11:33:39.924 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T11:33:39.925 INFO:teuthology.results:seed: '3643' 2021-11-12T02:19:26.838 INFO:root:teuthology version: 1.1.0-d4737010 2021-11-12T02:19:26.871 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-12T04:36:38.879 INFO:teuthology.results:Tests finished! gathering results... 2021-11-12T04:36:59.638 INFO:scrape:Found 220 jobs 2021-11-12T04:37:13.766 INFO:scrape:Found 5 distinct failure reasons 2021-11-12T04:37:13.767 INFO:scrape:Failure: "2021-11-12T00:02:13.738121+0000 mon.a (mon.0) 242 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2021-11-12T04:37:13.768 INFO:scrape:31 jobs 2021-11-12T04:37:13.768 INFO:scrape:suites intersection: ['conf'] 2021-11-12T04:37:13.769 INFO:scrape:suites union: ['clusters/3-node', 'clusters/fixed-3', 'conf', 'features/no-object-map', 'features/object-map', 'install/ceph', 'krbd/fsx/{ceph/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'krbd/rbd/{bluestore-bitmap', 'ms_mode$/{crc}', 'ms_mode$/{prefer-crc}', 'ms_mode/crc', 'ms_mode/legacy', 'ms_mode/secure', 'msgr-failures/few', 'msgr-failures/many', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'randomized-striping-on}', 'striping/default/{msgr-failures/many', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-1-client}', 'tasks/fsx-3-client}', 'tasks/krbd_data_pool}', 'tasks/krbd_huge_osdmap}', 'tasks/krbd_latest_osdmap_on_map}', 'tasks/krbd_namespaces}', 'tasks/krbd_udev_enumerate}', 'tasks/krbd_udev_netlink_enobufs}', 'tasks/krbd_udev_netns}', 'tasks/krbd_udev_symlinks}', 'tasks/rbd_huge_tickets}', 'tasks/rbd_image_read}', 'tasks/rbd_kernel}', 'tasks/rbd_map_snapshot_io}', 'tasks/rbd_simple_big}', 'tasks/rbd_workunit_kernel_untar_build}', 'tasks/rbd_workunit_suites_ffsb}', 'tasks/rbd_workunit_suites_fsstress_ext4}', 'tasks/rbd_workunit_suites_fsstress}', 'tasks/rbd_workunit_suites_fsx}', 'tasks/rbd_workunit_suites_iozone}', 'tasks/rbd_workunit_trivial_sync}'] 2021-11-12T04:37:13.769 INFO:scrape: 2021-11-12T04:37:13.770 INFO:scrape:Failure: Command failed on gibba037 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludefx34980b -r -- '-g auto -g blockdev -x clone'" 2021-11-12T04:37:13.770 INFO:scrape:2 jobs: ['6486199', '6486310'] 2021-11-12T04:37:13.771 INFO:scrape:suites intersection: ['conf', 'krbd/singleton/{bluestore-bitmap', 'ms_mode$/{crc}', 'tasks/rbd_xfstests}'] 2021-11-12T04:37:13.771 INFO:scrape:suites union: ['conf', 'krbd/singleton/{bluestore-bitmap', 'ms_mode$/{crc}', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_xfstests}'] 2021-11-12T04:37:13.772 INFO:scrape: 2021-11-12T04:37:13.772 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2021-11-12T04:37:13.773 INFO:scrape:3 jobs: ['6486356', '6486245', '6486200'] 2021-11-12T04:37:13.773 INFO:scrape:suites intersection: ['clusters/3-node', 'conf', 'krbd/fsx/{ceph/ceph', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-3-client}'] 2021-11-12T04:37:13.774 INFO:scrape:suites union: ['clusters/3-node', 'conf', 'features/no-object-map', 'features/object-map', 'krbd/fsx/{ceph/ceph', 'ms_mode$/{crc}', 'ms_mode$/{prefer-crc}', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-3-client}'] 2021-11-12T04:37:13.774 INFO:scrape: 2021-11-12T04:37:13.774 INFO:scrape:Failure: SSH connection to gibba029 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' 2021-11-12T04:37:13.775 INFO:scrape:2 jobs: ['6486257', '6486403'] 2021-11-12T04:37:13.775 INFO:scrape:suites intersection: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2021-11-12T04:37:13.776 INFO:scrape:suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/crc', 'ms_mode/secure', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2021-11-12T04:37:13.776 INFO:scrape: 2021-11-12T04:37:13.777 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major 2021-11-12T04:37:13.777 INFO:scrape:1 jobs: ['6486201'] 2021-11-12T04:37:13.777 INFO:scrape:suites: ['clusters/separate-client', 'conf', 'filestore-xfs', 'kernels/pre-single-major', 'krbd/unmap/{ceph/ceph', 'tasks/unmap}'] 2021-11-12T04:37:13.778 INFO:scrape: 2021-11-12T04:37:13.785 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/ failed: 36 dead: 3 running: 0 waiting: 0 queued: 0 passed: 181 Fail ================================================================= [6486199] krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} ----------------------------------------------------------------- time: 03:43:21 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486199/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486199/ sentry: https://sentry.ceph.com/organizations/ceph/?query=19fff098316d46b89a344807d78d73a3 Command failed on gibba037 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludefx34980b -r -- '-g auto -g blockdev -x clone'" [6486201] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486201/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486201/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a16153c38ac7493b861eb5e7b2aab026 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&re f=nightly_pre-single-major [6486204] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:17:48 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486204/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486204/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5a20df2138bf42da8602d45b2d8a2d75 "2021-11-11T12:52:12.229445+0000 mon.b (mon.0) 113 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486210] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:16:43 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486210/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486210/ "2021-11-11T13:12:40.059661+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486211] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} ----------------------------------------------------------------- time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486211/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486211/ sentry: https://sentry.ceph.com/organizations/ceph/?query=abe1eb64ff5b491383332d58b5001901 "2021-11-11T13:04:49.657369+0000 mon.b (mon.0) 325 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486222] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} ----------------------------------------------------------------- time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486222/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486222/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7979061371ab4cc2a46629b4df9e5eaf "2021-11-11T14:33:53.528293+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486223] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} ----------------------------------------------------------------- time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486223/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486223/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ead62e8c98994d129a7d34e0f1151868 "2021-11-11T14:13:31.548974+0000 mon.a (mon.0) 222 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log [6486245] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486245/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486245/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5186cf56934c4128ae368750330951a8 reached maximum tries (90) after waiting for 540 seconds [6486251] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} ----------------------------------------------------------------- time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486251/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486251/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38ddeba140f34526bd82978f025ec5fe "2021-11-11T16:36:31.030983+0000 mon.a (mon.0) 286 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486267] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} ----------------------------------------------------------------- time: 00:17:43 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486267/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486267/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d228524b80444f42a763c2079fdcde0b "2021-11-11T17:38:39.713813+0000 mon.a (mon.0) 276 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486278] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_huge_osdmap} ----------------------------------------------------------------- time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486278/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486278/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8798b3c520d24ad79c57957f1e7d2b6c "2021-11-11T17:58:40.788066+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486285] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:18:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486285/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486285/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e00087ca51834742affd1e6d2b4ab3f1 "2021-11-11T18:41:27.841386+0000 mon.b (mon.0) 355 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log [6486287] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} ----------------------------------------------------------------- time: 00:17:39 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486287/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486287/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c809d4284d164d05a3c893f0f7eca1d9 "2021-11-11T18:54:03.462371+0000 mon.a (mon.0) 233 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log [6486291] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:22:40 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486291/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486291/ "2021-11-11T19:15:18.487215+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486292] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} ----------------------------------------------------------------- time: 00:18:16 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486292/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486292/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6d4f1c96bf44c609619e9001034d24d "2021-11-11T19:02:24.310684+0000 mon.b (mon.0) 200 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486300] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} ----------------------------------------------------------------- time: 00:18:03 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486300/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486300/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8d50ad7ff484ec6a307684ae8d75308 "2021-11-11T19:28:48.457568+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486301] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:17:37 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486301/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486301/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dfa075597abf4e90a0ea21122fb9ddf9 "2021-11-11T19:51:15.670298+0000 mon.b (mon.0) 255 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486309] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} ----------------------------------------------------------------- time: 00:17:52 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486309/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486309/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f5c7928c54e412884b9a7312ceef45c "2021-11-11T20:14:33.206482+0000 mon.a (mon.0) 228 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486310] krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/many tasks/rbd_xfstests} ----------------------------------------------------------------- time: 03:57:54 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486310/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486310/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f052c6f669fc4e2da3b9347d9e3f47f2 Command failed on gibba005 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludedw81x6wj -r -- '-g auto -g blockdev -x clone'" [6486321] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:17:39 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486321/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486321/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a2de9e5b0ad4b3b9bbcd729603846fe "2021-11-11T20:58:31.527608+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486322] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} ----------------------------------------------------------------- time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486322/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486322/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c470ee9d0b548bebe52e9641430e00b "2021-11-11T21:08:25.748745+0000 mon.a (mon.0) 297 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486324] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486324/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486324/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e779266ddea4dc692529faf4f94223f "2021-11-11T21:14:29.729406+0000 mon.b (mon.0) 333 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486325] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} ----------------------------------------------------------------- time: 00:17:57 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486325/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486325/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1359bc5319b54a7cbb5d80145b9e9dd8 "2021-11-11T21:10:41.753142+0000 mon.a (mon.0) 161 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486327] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:17:58 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486327/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486327/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad06447883324974a5237b313661cc9b "2021-11-11T21:29:58.382101+0000 mon.a (mon.0) 231 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486342] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} ----------------------------------------------------------------- time: 00:17:35 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486342/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486342/ sentry: https://sentry.ceph.com/organizations/ceph/?query=658bd355d4da444a9b2fd680487fe194 "2021-11-11T22:31:49.607449+0000 mon.a (mon.0) 217 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486346] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} ----------------------------------------------------------------- time: 00:18:03 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486346/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486346/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6c88ed9b820f490788c33a89ed06524c "2021-11-11T22:43:03.131888+0000 mon.a (mon.0) 266 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486356] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:16:57 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486356/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486356/ sentry: https://sentry.ceph.com/organizations/ceph/?query=21f627871b8e44fe8a8a0b81daa974ef reached maximum tries (90) after waiting for 540 seconds [6486366] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} ----------------------------------------------------------------- time: 00:17:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486366/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486366/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dab26127957c4836a3d7c9bb6954a25b "2021-11-12T00:02:13.738121+0000 mon.a (mon.0) 242 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486367] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} ----------------------------------------------------------------- time: 00:18:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486367/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486367/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4708eec6e6114ec8b22ec4ea04478aaf "2021-11-11T23:51:56.594330+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486375] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} ----------------------------------------------------------------- time: 00:18:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486375/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486375/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7b508f478464ed995e81e515d0771d7 "2021-11-12T00:33:11.439547+0000 mon.a (mon.0) 332 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log [6486381] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:16:53 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486381/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486381/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5bb7e28aebc4eccbb881f2fcad745d3 "2021-11-12T00:44:42.825387+0000 mon.a (mon.0) 113 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6486383] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} ----------------------------------------------------------------- time: 00:17:45 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486383/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486383/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b85ff2f9153949e2ab9b67d593c98e04 "2021-11-12T01:03:45.662864+0000 mon.a (mon.0) 227 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486389] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} ----------------------------------------------------------------- time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486389/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486389/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3b7fb32b57f4f7c9bc46276d902a072 "2021-11-12T01:29:01.752253+0000 mon.a (mon.0) 201 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486395] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} ----------------------------------------------------------------- time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486395/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486395/ sentry: https://sentry.ceph.com/organizations/ceph/?query=07937f51e6b148d4a7b649ae9bae2cef "2021-11-12T01:43:51.137886+0000 mon.a (mon.0) 205 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6486406] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} ----------------------------------------------------------------- time: 00:17:53 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486406/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486406/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5341112aaa141d2b62a2bef2d1fbe5b "2021-11-12T02:16:36.724423+0000 mon.a (mon.0) 236 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log [6486411] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} ----------------------------------------------------------------- time: 00:17:52 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486411/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486411/ sentry: https://sentry.ceph.com/organizations/ceph/?query=92a706d13f644f66b08b1774dc01dca5 "2021-11-12T02:40:42.969845+0000 mon.a (mon.0) 222 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log Dead ================================================================= [6486200] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486200/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486200/ sentry: https://sentry.ceph.com/organizations/ceph/?query=73d9547947a24449b20469d1ef0f15ab reached maximum tries (100) after waiting for 600 seconds [6486257] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 02:26:26 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486257/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486257/ sentry: https://sentry.ceph.com/organizations/ceph/?query=496d73ce56c940f8be9dc79dda6944e0 SSH connection to gibba029 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' [6486403] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 02:38:12 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486403/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486403/ sentry: https://sentry.ceph.com/organizations/ceph/?query=42facdb288134df1a3ff8652fc109c2c SSH connection to gibba038 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' Pass ================================================================= [6486195] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486195/ [6486196] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:20:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486196/ [6486197] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} time: 00:25:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486197/ [6486198] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} time: 00:16:52 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486198/ [6486202] krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} time: 00:07:19 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486202/ [6486203] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486203/ [6486205] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} time: 00:10:02 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486205/ [6486206] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_huge_osdmap} time: 00:10:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486206/ [6486207] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:36:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486207/ [6486208] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486208/ [6486209] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486209/ [6486212] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:08:46 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486212/ [6486213] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} time: 00:20:07 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486213/ [6486214] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486214/ [6486215] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486215/ [6486216] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486216/ [6486217] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} time: 00:07:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486217/ [6486218] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:24:41 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486218/ [6486219] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} time: 00:10:13 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486219/ [6486220] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:09:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486220/ [6486221] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486221/ [6486224] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} time: 00:09:22 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486224/ [6486225] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_modprobe} time: 00:07:27 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486225/ [6486226] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486226/ [6486227] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:02:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486227/ [6486228] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486228/ [6486229] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:18:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486229/ [6486230] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486230/ [6486231] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486231/ [6486232] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:11:21 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486232/ [6486233] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} time: 00:26:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486233/ [6486234] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} time: 00:07:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486234/ [6486235] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:23:56 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486235/ [6486236] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} time: 00:25:19 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486236/ [6486237] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} time: 00:20:04 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486237/ [6486238] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486238/ [6486239] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:15:07 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486239/ [6486240] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} time: 00:10:27 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486240/ [6486241] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} time: 00:10:39 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486241/ [6486242] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:37:31 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486242/ [6486243] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486243/ [6486244] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486244/ [6486246] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} time: 00:10:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486246/ [6486247] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:01:11 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486247/ [6486248] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} time: 00:22:13 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486248/ [6486249] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486249/ [6486250] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:13:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486250/ [6486252] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} time: 00:07:28 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486252/ [6486253] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:19:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486253/ [6486254] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/pggrow thrashosds-health workloads/rbd_fio} time: 00:39:48 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486254/ [6486255] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} time: 00:09:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486255/ [6486256] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:09:26 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486256/ [6486258] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} time: 00:08:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486258/ [6486259] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:25:14 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486259/ [6486260] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} time: 00:09:25 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486260/ [6486261] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} time: 00:08:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486261/ [6486262] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486262/ [6486263] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486263/ [6486264] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486264/ [6486265] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486265/ [6486266] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486266/ [6486268] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} time: 00:29:10 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486268/ [6486269] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486269/ [6486270] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:23:25 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486270/ [6486271] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486271/ [6486272] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} time: 00:16:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486272/ [6486273] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486273/ [6486274] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} time: 00:19:54 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486274/ [6486275] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} time: 00:10:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486275/ [6486276] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486276/ [6486277] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486277/ [6486279] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:36:37 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486279/ [6486280] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486280/ [6486281] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486281/ [6486282] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:17:46 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486282/ [6486283] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:02:03 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486283/ [6486284] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} time: 00:10:14 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486284/ [6486286] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} time: 00:20:09 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486286/ [6486288] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:42 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486288/ [6486289] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486289/ [6486290] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} time: 00:07:48 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486290/ [6486293] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:09:45 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486293/ [6486294] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486294/ [6486295] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486295/ [6486296] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:23:12 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486296/ [6486297] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} time: 00:09:22 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486297/ [6486298] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} time: 00:07:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486298/ [6486299] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:20:13 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486299/ [6486302] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:35 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486302/ [6486303] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486303/ [6486304] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:10:51 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486304/ [6486305] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} time: 00:26:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486305/ [6486306] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486306/ [6486307] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:28:33 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486307/ [6486308] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} time: 00:25:08 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486308/ [6486311] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:02:21 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486311/ [6486312] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486312/ [6486313] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:15:35 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486313/ [6486314] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486314/ [6486315] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_huge_osdmap} time: 00:10:09 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486315/ [6486316] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:37:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486316/ [6486317] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486317/ [6486318] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} time: 00:07:43 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486318/ [6486319] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:15:50 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486319/ [6486320] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486320/ [6486323] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486323/ [6486326] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} time: 00:08:19 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486326/ [6486328] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486328/ [6486329] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:09:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486329/ [6486330] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} time: 00:14:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486330/ [6486331] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} time: 00:08:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486331/ [6486332] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:24:46 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486332/ [6486333] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} time: 00:10:12 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486333/ [6486334] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} time: 00:07:40 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486334/ [6486335] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:14:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486335/ [6486336] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} time: 00:37:22 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486336/ [6486337] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486337/ [6486338] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486338/ [6486339] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486339/ [6486340] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486340/ [6486341] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:13:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486341/ [6486343] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} time: 00:08:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486343/ [6486344] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:28:29 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486344/ [6486345] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} time: 00:25:18 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486345/ [6486347] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} time: 00:08:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486347/ [6486348] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} time: 00:20:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486348/ [6486349] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486349/ [6486350] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} time: 00:16:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486350/ [6486351] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486351/ [6486352] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486352/ [6486353] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:36:58 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486353/ [6486354] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:12:25 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486354/ [6486355] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486355/ [6486357] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} time: 00:10:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486357/ [6486358] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:09:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486358/ [6486359] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} time: 00:20:03 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486359/ [6486360] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486360/ [6486361] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:37 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486361/ [6486362] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} time: 00:10:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486362/ [6486363] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} time: 00:07:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486363/ [6486364] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:26:06 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486364/ [6486365] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:01:22 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486365/ [6486368] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486368/ [6486369] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} time: 00:08:49 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486369/ [6486370] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:23:10 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486370/ [6486371] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486371/ [6486372] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_modprobe} time: 00:07:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486372/ [6486373] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:17:28 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486373/ [6486374] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486374/ [6486376] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:42 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486376/ [6486377] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486377/ [6486378] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486378/ [6486379] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} time: 00:25:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486379/ [6486380] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} time: 00:07:23 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486380/ [6486382] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} time: 00:25:22 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486382/ [6486384] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486384/ [6486385] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486385/ [6486386] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} time: 00:10:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486386/ [6486387] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_huge_osdmap} time: 00:10:16 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486387/ [6486388] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:38:00 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486388/ [6486390] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486390/ [6486391] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486391/ [6486392] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/upmap thrashosds-health workloads/rbd_fio} time: 00:39:30 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486392/ [6486393] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} time: 00:10:42 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486393/ [6486394] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:01:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486394/ [6486396] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:12:49 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486396/ [6486397] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486397/ [6486398] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} time: 00:09:49 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486398/ [6486399] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} time: 00:08:17 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486399/ [6486400] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:26:07 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486400/ [6486401] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} time: 00:10:05 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486401/ [6486402] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:09:28 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486402/ [6486404] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486404/ [6486405] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:24:49 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486405/ [6486407] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} time: 00:07:44 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486407/ [6486408] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486408/ [6486409] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} time: 00:09:15 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486409/ [6486410] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:17:09 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486410/ [6486412] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} time: 00:13:43 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486412/ [6486413] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486413/ [6486414] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} time: 00:28:59 info: http://pulpito.ceph.com/teuthology-2021-11-05_03:20:01-krbd-master-testing-default-gibba/6486414/