2021-11-21T08:09:26.382 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-21T08:09:26.384 INFO:teuthology.results:seed: '7157' 2021-11-21T18:27:38.760 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-21T18:27:38.816 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-21T19:36:43.984 INFO:teuthology.results:Tests finished! gathering results... 2021-11-21T19:37:00.546 INFO:scrape:Found 220 jobs 2021-11-21T19:37:03.969 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512668/teuthology.log 2021-11-21T19:37:10.356 INFO:scrape:Found 7 distinct failure reasons 2021-11-21T19:37:10.357 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2021-11-21T19:37:10.358 INFO:scrape:2 jobs: ['6512587', '6512698'] 2021-11-21T19:37:10.358 INFO:scrape:suites intersection: ['clusters/3-node', 'conf', 'krbd/fsx/{ceph/ceph', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-1-client}'] 2021-11-21T19:37:10.359 INFO:scrape:suites union: ['clusters/3-node', 'conf', 'features/no-object-map', 'features/object-map', 'krbd/fsx/{ceph/ceph', 'ms_mode$/{legacy}', 'ms_mode$/{secure}', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-1-client}'] 2021-11-21T19:37:10.359 INFO:scrape: 2021-11-21T19:37:10.359 INFO:scrape:Failure: "2021-11-21T12:57:04.184467+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log 2021-11-21T19:37:10.360 INFO:scrape:33 jobs 2021-11-21T19:37:10.361 INFO:scrape:suites intersection: ['conf'] 2021-11-21T19:37:10.361 INFO:scrape:suites union: ['clusters/3-node', 'clusters/fixed-3', 'conf', 'features/no-object-map', 'features/object-map', 'install/ceph', 'krbd/fsx/{ceph/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'krbd/rbd/{bluestore-bitmap', 'ms_mode$/{crc}', 'ms_mode$/{prefer-crc}', 'ms_mode$/{secure}', 'ms_mode/crc', 'ms_mode/legacy', 'ms_mode/secure', 'msgr-failures/few', 'msgr-failures/many', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'randomized-striping-on}', 'striping/default/{msgr-failures/few', 'striping/default/{msgr-failures/many', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-1-client}', 'tasks/fsx-3-client}', 'tasks/krbd_data_pool}', 'tasks/krbd_huge_osdmap}', 'tasks/krbd_latest_osdmap_on_map}', 'tasks/krbd_namespaces}', 'tasks/krbd_udev_enumerate}', 'tasks/krbd_udev_netlink_enobufs}', 'tasks/krbd_udev_netns}', 'tasks/krbd_udev_symlinks}', 'tasks/rbd_concurrent}', 'tasks/rbd_fio}', 'tasks/rbd_huge_tickets}', 'tasks/rbd_image_read}', 'tasks/rbd_kernel}', 'tasks/rbd_workunit_kernel_untar_build}', 'tasks/rbd_workunit_suites_dbench}', 'tasks/rbd_workunit_suites_ffsb}', 'tasks/rbd_workunit_suites_fsstress_ext4}', 'tasks/rbd_workunit_suites_fsx}', 'tasks/rbd_workunit_trivial_sync}'] 2021-11-21T19:37:10.362 INFO:scrape: 2021-11-21T19:37:10.362 INFO:scrape:Dead: None 2021-11-21T19:37:10.363 INFO:scrape:2 jobs: ['6512668', '6512713'] 2021-11-21T19:37:10.363 INFO:scrape: 2021-11-21T19:37:10.364 INFO:scrape:Failure: Command failed on gibba016 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeww6it24p -r -- '-g auto -g blockdev -x clone'" 2021-11-21T19:37:10.364 INFO:scrape:2 jobs: ['6512516', '6512627'] 2021-11-21T19:37:10.364 INFO:scrape:suites intersection: ['conf', 'krbd/singleton/{bluestore-bitmap', 'tasks/rbd_xfstests}'] 2021-11-21T19:37:10.365 INFO:scrape:suites union: ['conf', 'krbd/singleton/{bluestore-bitmap', 'ms_mode$/{crc}', 'ms_mode$/{secure}', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_xfstests}'] 2021-11-21T19:37:10.365 INFO:scrape: 2021-11-21T19:37:10.366 INFO:scrape:Failure: Failure object was: {'gibba031.front.sepia.ceph.com': {'cache_updated': False, 'stdout': '', 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['apt', 'apache2'], 'autoclean': False, 'install_recommends': None, 'name': ['apt', 'apache2'], 'purge': False, 'allow_unauthenticated': False, 'state': 'latest', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1637504177, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'apache2\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stdout_lines': [], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time') 2021-11-21T19:37:10.366 INFO:scrape:1 jobs: ['6512637'] 2021-11-21T19:37:10.367 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/secure', 'msgr-failures/few', 'tasks/krbd_namespaces}'] 2021-11-21T19:37:10.368 INFO:scrape: 2021-11-21T19:37:10.368 INFO:scrape:Failure: "2021-11-21T15:53:59.600910+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.9533.0:11518394 1.6 1:6a7654a4:::rbd_header.2f499650ec18:head [watch reconnect cookie 18446462598732841104 gen 130136] snapc 0=[] ondisk+write+known_if_redirected e19) initiated 2021-11-21T15:53:29.538823+0000 currently started" in cluster log 2021-11-21T19:37:10.368 INFO:scrape:1 jobs: ['6512647'] 2021-11-21T19:37:10.369 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/legacy', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2021-11-21T19:37:10.369 INFO:scrape: 2021-11-21T19:37:10.370 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major 2021-11-21T19:37:10.370 INFO:scrape:1 jobs: ['6512518'] 2021-11-21T19:37:10.371 INFO:scrape:suites: ['clusters/separate-client', 'conf', 'filestore-xfs', 'kernels/pre-single-major', 'krbd/unmap/{ceph/ceph', 'tasks/unmap}'] 2021-11-21T19:37:10.371 INFO:scrape: 2021-11-21T19:37:10.378 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/ failed: 39 dead: 3 running: 0 waiting: 0 queued: 0 passed: 178 Fail ================================================================= [6512516] krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} ----------------------------------------------------------------- time: 03:39:26 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512516/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512516/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5579c860431c4397a89fc20402592f7f Command failed on gibba016 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeww6it24p -r -- '-g auto -g blockdev -x clone'" [6512518] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512518/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512518/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f1d956dfc83d4df097a5971749ca3242 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&re f=nightly_pre-single-major [6512521] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512521/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512521/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4a1a6e12c5fe4381a3a37a984112f303 "2021-11-21T08:45:08.957389+0000 mon.b (mon.0) 160 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512523] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_huge_osdmap} ----------------------------------------------------------------- time: 00:17:03 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512523/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512523/ sentry: https://sentry.ceph.com/organizations/ceph/?query=83994d6139c14f3a95553cebd44e9fa7 "2021-11-21T08:57:12.510671+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512528] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} ----------------------------------------------------------------- time: 00:17:03 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512528/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512528/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a36d94a7fb1a4804809049e0d97acc01 "2021-11-21T09:54:12.099400+0000 mon.b (mon.0) 244 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log [6512536] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} ----------------------------------------------------------------- time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512536/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512536/ sentry: https://sentry.ceph.com/organizations/ceph/?query=64e47a5d2a5644b680a91b66bfc58cfb "2021-11-21T09:41:50.556690+0000 mon.a (mon.0) 299 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512537] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} ----------------------------------------------------------------- time: 00:17:05 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512537/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512537/ sentry: https://sentry.ceph.com/organizations/ceph/?query=07cc3760814b40a6a1134afb646f5a95 "2021-11-21T09:46:54.311039+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512554] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} ----------------------------------------------------------------- time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512554/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512554/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2ba0712a0fd449fdbdb27cd7d1698234 "2021-11-21T10:25:07.318116+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512562] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512562/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512562/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8934eca9dd4141f7adfe6310aaa772f4 "2021-11-21T10:52:24.348814+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512565] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} ----------------------------------------------------------------- time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512565/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512565/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03357f25461741aeb7fa2138241691f8 "2021-11-21T10:58:38.568527+0000 mon.a (mon.0) 129 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512568] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} ----------------------------------------------------------------- time: 00:16:52 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512568/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512568/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3dc0a590b7e5479fa697c9814826984d "2021-11-21T11:15:32.259145+0000 mon.a (mon.0) 154 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512576] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} ----------------------------------------------------------------- time: 00:17:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512576/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512576/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f063be4521064ea6bd0f56e1f496b12f "2021-11-21T11:52:30.170488+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512587] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:16:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512587/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512587/ sentry: https://sentry.ceph.com/organizations/ceph/?query=94b41ca1afdb405589670336c0bfa851 reached maximum tries (90) after waiting for 540 seconds [6512593] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:17:13 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512593/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512593/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58440e38a3494b5b821b26be159fb033 "2021-11-21T12:25:37.044492+0000 mon.a (mon.0) 128 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512595] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_huge_osdmap} ----------------------------------------------------------------- time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512595/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512595/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c0744e7774084f74b2e5796b17fe80c3 "2021-11-21T12:29:14.734767+0000 mon.a (mon.0) 111 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512602] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} ----------------------------------------------------------------- time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512602/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512602/ sentry: https://sentry.ceph.com/organizations/ceph/?query=909decf7e0724769b2e55b76fcdf00f9 "2021-11-21T12:56:26.843778+0000 mon.a (mon.0) 231 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512604] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} ----------------------------------------------------------------- time: 00:17:03 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512604/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512604/ sentry: https://sentry.ceph.com/organizations/ceph/?query=35f12d89e5c34935a3b2e1a6823bc76b "2021-11-21T13:04:59.844311+0000 mon.a (mon.0) 318 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512608] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:35:24 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512608/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512608/ "2021-11-21T12:57:04.184467+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512612] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} ----------------------------------------------------------------- time: 00:17:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512612/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512612/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7ebc837a0644465a90bfe66d8f7367e "2021-11-21T13:36:01.427096+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512617] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} ----------------------------------------------------------------- time: 00:16:51 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512617/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512617/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48133f57466040fbb66d8d9955afba6e "2021-11-21T13:19:10.221420+0000 mon.b (mon.0) 139 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512625] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} ----------------------------------------------------------------- time: 00:17:27 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512625/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512625/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aec4d99136ee437c83f8769aa2e9b903 "2021-11-21T13:57:24.718606+0000 mon.a (mon.0) 204 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512627] krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/many tasks/rbd_xfstests} ----------------------------------------------------------------- time: 04:04:33 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512627/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512627/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ab9f2dd61b04a87a1d4dc1d63bff2f1 Command failed on gibba039 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude5o1uf989 -r -- '-g auto -g blockdev -x clone'" [6512633] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} ----------------------------------------------------------------- time: 00:16:57 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512633/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512633/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc594ed5a93341c4b1b114020951e1fd "2021-11-21T13:58:13.403896+0000 mon.b (mon.0) 124 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [6512647] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 02:00:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512647/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512647/ "2021-11-21T15:53:59.600910+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.9533.0:11518394 1.6 1:6a7654a4:::rbd_header.2f499650ec18:head [watch reconnect cookie 18446462598732841104 gen 130136] snapc 0=[] ondisk+write+known_if_redirected e19) initiated 2021-11-21T15:53:29.538823+0000 currently started" in cluster log [6512650] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} ----------------------------------------------------------------- time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512650/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512650/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7f1078b636a944aa974541a7e0d691ea "2021-11-21T15:08:00.301497+0000 mon.a (mon.0) 337 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512652] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512652/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512652/ "2021-11-21T15:07:23.199820+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512654] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} ----------------------------------------------------------------- time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512654/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512654/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cabc278a985f49bebb059b7b94cc0943 "2021-11-21T15:17:42.976556+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512658] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} ----------------------------------------------------------------- time: 00:16:54 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512658/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512658/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a80f54dc5d4943568997955578113b61 "2021-11-21T15:25:33.462730+0000 mon.b (mon.0) 109 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6512667] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512667/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512667/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5aaa76b5ca9242ba8c1705fffb5e0a46 "2021-11-21T15:57:23.945322+0000 mon.b (mon.0) 171 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512669] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} ----------------------------------------------------------------- time: 00:17:08 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512669/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512669/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2cce707f45134743ac1e54285d2cd05c "2021-11-21T16:04:04.070556+0000 mon.b (mon.0) 176 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512670] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} ----------------------------------------------------------------- time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512670/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512670/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28c9887afa384ed3b8f840d0dff699b3 "2021-11-21T16:15:12.007229+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512677] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} ----------------------------------------------------------------- time: 00:17:05 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512677/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512677/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b579d7430df472a86e93527996885b3 "2021-11-21T16:31:07.532398+0000 mon.a (mon.0) 278 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512684] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} ----------------------------------------------------------------- time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512684/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512684/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2471ba09ba30469eacd44d528092d281 "2021-11-21T16:47:33.354119+0000 mon.a (mon.0) 230 : cluster [WRN] Health check failed: 4 osds down (OSD_DOWN)" in cluster log [6512698] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512698/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7f1a91d0605f49cba520fc2b237bae3e reached maximum tries (90) after waiting for 540 seconds [6512699] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} ----------------------------------------------------------------- time: 00:17:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512699/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512699/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e0f1d72cd42b4636b8924d4a277217e3 "2021-11-21T17:32:41.638526+0000 mon.b (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512706] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} ----------------------------------------------------------------- time: 00:17:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512706/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512706/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6005c29546ef4a8c9b7907f052aa732b "2021-11-21T17:58:53.614658+0000 mon.a (mon.0) 270 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512720] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 00:16:50 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512720/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512720/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6448e4198f3426db6178a0b0cd5758a "2021-11-21T18:16:15.725197+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [6512723] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} ----------------------------------------------------------------- time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512723/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512723/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cca72a742bdb4183b46a4b0864f8eaaf "2021-11-21T18:28:02.395547+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6512730] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} ----------------------------------------------------------------- time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512730/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512730/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fae936d267c64d92ad718e42af2388ae "2021-11-21T18:48:15.680346+0000 mon.b (mon.0) 296 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log Dead ================================================================= [6512637] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} ----------------------------------------------------------------- time: 00:03:16 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512637/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512637/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f0fb870cb26b4ada9eaa8ee185843100 Failure object was: {'gibba031.front.sepia.ceph.com': {'cache_updated': False, 'stdout': '', 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['apt', 'apache2'], 'autoclean': False, 'install_recommends': None, 'name': ['apt', 'apache2'], 'purge': False, 'allow_unauthenticated': False, 'state': 'latest', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1637504177, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'apache2\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stdout_lines': [], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/py thon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworke r/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/ virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.co m_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/py thon3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teu thology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/s ite-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/ src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/vi rtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d 5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com _git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/pyt hon3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuth worker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb8 8356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d 5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git _teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3 .6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.c eph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/ lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time') [6512668] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512668/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512668/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [6512713] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512713/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512713/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds Pass ================================================================= [6512512] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} time: 00:07:25 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512512/ [6512513] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:20:15 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512513/ [6512514] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} time: 00:24:32 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512514/ [6512515] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} time: 00:15:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512515/ [6512517] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} time: 00:39:45 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512517/ [6512519] krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} time: 00:07:17 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512519/ [6512520] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} time: 00:09:19 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512520/ [6512522] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512522/ [6512524] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:35:44 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512524/ [6512525] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:11:43 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512525/ [6512526] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} time: 00:07:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512526/ [6512527] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:15:50 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512527/ [6512529] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:07:32 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512529/ [6512530] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512530/ [6512531] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:12:34 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512531/ [6512532] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512532/ [6512533] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512533/ [6512534] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} time: 00:06:53 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512534/ [6512535] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:23:41 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512535/ [6512538] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512538/ [6512539] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512539/ [6512540] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:20:42 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512540/ [6512541] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} time: 00:08:37 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512541/ [6512542] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_modprobe} time: 00:06:47 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512542/ [6512543] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512543/ [6512544] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 00:59:21 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512544/ [6512545] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} time: 00:08:14 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512545/ [6512546] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:18:05 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512546/ [6512547] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:02 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512547/ [6512548] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512548/ [6512549] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:10:30 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512549/ [6512550] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} time: 00:25:24 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512550/ [6512551] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} time: 00:06:47 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512551/ [6512552] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:23:13 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512552/ [6512553] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} time: 00:24:50 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512553/ [6512555] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512555/ [6512556] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512556/ [6512557] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} time: 00:09:15 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512557/ [6512558] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512558/ [6512559] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:37:15 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512559/ [6512560] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512560/ [6512561] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} time: 00:07:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512561/ [6512563] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} time: 00:09:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512563/ [6512564] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:00:46 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512564/ [6512566] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:12:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512566/ [6512567] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512567/ [6512569] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512569/ [6512570] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512570/ [6512571] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/pggrow thrashosds-health workloads/rbd_fio} time: 00:38:44 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512571/ [6512572] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512572/ [6512573] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512573/ [6512574] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512574/ [6512575] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} time: 00:08:28 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512575/ [6512577] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} time: 00:09:17 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512577/ [6512578] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} time: 00:07:07 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512578/ [6512579] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:18:07 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512579/ [6512580] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512580/ [6512581] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:16:18 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512581/ [6512582] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:08:19 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512582/ [6512583] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512583/ [6512584] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:12:13 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512584/ [6512585] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} time: 00:27:54 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512585/ [6512586] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512586/ [6512588] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} time: 00:24:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512588/ [6512589] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512589/ [6512590] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} time: 00:07:24 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512590/ [6512591] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512591/ [6512592] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512592/ [6512594] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512594/ [6512596] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:35:35 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512596/ [6512597] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512597/ [6512598] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} time: 00:07:46 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512598/ [6512599] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:16:37 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512599/ [6512600] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:00:40 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512600/ [6512601] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512601/ [6512603] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} time: 00:19:29 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512603/ [6512605] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512605/ [6512606] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} time: 00:09:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512606/ [6512607] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512607/ [6512609] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512609/ [6512610] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:49 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512610/ [6512611] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} time: 00:12:35 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512611/ [6512613] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:20:45 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512613/ [6512614] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} time: 00:08:56 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512614/ [6512615] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} time: 00:06:44 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512615/ [6512616] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:19:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512616/ [6512618] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:19:13 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512618/ [6512619] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512619/ [6512620] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512620/ [6512621] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:10:35 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512621/ [6512622] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} time: 00:28:21 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512622/ [6512623] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} time: 00:07:07 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512623/ [6512624] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512624/ [6512626] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512626/ [6512628] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 00:59:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512628/ [6512629] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} time: 00:09:25 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512629/ [6512630] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:14:25 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512630/ [6512631] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512631/ [6512632] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_huge_osdmap} time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512632/ [6512634] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:11:32 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512634/ [6512635] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} time: 00:07:24 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512635/ [6512636] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:14:33 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512636/ [6512638] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 00:59:22 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512638/ [6512639] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} time: 00:21:47 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512639/ [6512640] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:12:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512640/ [6512641] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512641/ [6512642] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} time: 00:09:30 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512642/ [6512643] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512643/ [6512644] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:28:18 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512644/ [6512645] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512645/ [6512646] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512646/ [6512648] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512648/ [6512649] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:22:49 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512649/ [6512651] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} time: 00:07:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512651/ [6512653] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} time: 00:38:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512653/ [6512655] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:16:28 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512655/ [6512656] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:07:54 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512656/ [6512657] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} time: 00:12:43 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512657/ [6512659] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} time: 00:27:03 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512659/ [6512660] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} time: 00:07:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512660/ [6512661] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512661/ [6512662] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} time: 00:24:57 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512662/ [6512663] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} time: 00:15:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512663/ [6512664] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512664/ [6512665] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} time: 00:19:23 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512665/ [6512666] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512666/ [6512671] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512671/ [6512672] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} time: 00:07:43 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512672/ [6512673] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:14:34 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512673/ [6512674] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} time: 00:09:38 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512674/ [6512675] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:07:05 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512675/ [6512676] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} time: 00:19:28 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512676/ [6512678] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:10:27 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512678/ [6512679] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} time: 00:08:50 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512679/ [6512680] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} time: 00:06:57 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512680/ [6512681] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:24:19 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512681/ [6512682] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:04:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512682/ [6512683] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} time: 00:09:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512683/ [6512685] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512685/ [6512686] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} time: 00:08:21 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512686/ [6512687] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512687/ [6512688] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} time: 00:08:36 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512688/ [6512689] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_modprobe} time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512689/ [6512690] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512690/ [6512691] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} time: 00:08:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512691/ [6512692] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:17:27 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512692/ [6512693] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:08:06 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512693/ [6512694] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512694/ [6512695] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:10:31 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512695/ [6512696] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512696/ [6512697] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} time: 00:07:17 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512697/ [6512700] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} time: 00:18:46 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512700/ [6512701] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} time: 00:09:24 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512701/ [6512702] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512702/ [6512703] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} time: 00:09:51 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512703/ [6512704] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_huge_osdmap} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512704/ [6512705] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:36:53 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512705/ [6512707] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} time: 00:06:43 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512707/ [6512708] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:14:42 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512708/ [6512709] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/upmap thrashosds-health workloads/rbd_fio} time: 00:37:36 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512709/ [6512710] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} time: 00:08:55 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512710/ [6512711] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 00:58:57 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512711/ [6512712] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} time: 00:24:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512712/ [6512714] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512714/ [6512715] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512715/ [6512716] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} time: 00:07:11 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512716/ [6512717] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:24:51 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512717/ [6512718] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} time: 00:09:40 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512718/ [6512719] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:15 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512719/ [6512721] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512721/ [6512722] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:24:31 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512722/ [6512724] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} time: 00:07:10 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512724/ [6512725] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:16:09 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512725/ [6512726] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} time: 00:08:13 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512726/ [6512727] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:15:26 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512727/ [6512728] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:08:15 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512728/ [6512729] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512729/ [6512731] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} time: 00:28:34 info: http://pulpito.ceph.com/teuthology-2021-11-19_03:20:03-krbd-master-testing-default-gibba/6512731/