2022-07-29T22:47:36.203 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-07-29T22:47:36.206 INFO:teuthology.results:seed: '2617' 2022-07-29T22:52:49.326 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-07-29T22:52:49.332 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-30T00:33:57.644 INFO:teuthology.results:Tests finished! gathering results... 2022-07-30T00:33:58.929 INFO:scrape:Found 12 jobs 2022-07-30T00:34:01.801 INFO:scrape:Found 2 distinct failure reasons 2022-07-30T00:34:01.802 INFO:scrape:Failure: Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2022-07-30T00:34:01.803 INFO:scrape:3 jobs: ['6953937', '6953941', '6953945'] 2022-07-30T00:34:01.803 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/misc}}'] 2022-07-30T00:34:01.804 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/volumes/{overrides', 'test/misc}}'] 2022-07-30T00:34:01.804 INFO:scrape: 2022-07-30T00:34:01.805 INFO:scrape:Failure: "2022-07-29T23:37:28.025217+0000 mon.a (mon.0) 3196 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log 2022-07-30T00:34:01.806 INFO:scrape:1 jobs: ['6953946'] 2022-07-30T00:34:01.806 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/snapshot}}'] 2022-07-30T00:34:01.807 INFO:scrape: 2022-07-30T00:34:01.808 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [6953937] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:17:16 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953937/ log: http://qa-proxy.ceph.com/teuthology/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953937/ sentry: https://sentry.ceph.com/organizations/ceph/?query=487dab5d4388463189c0271a75be599b Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) [6953941] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:11:27 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953941/ log: http://qa-proxy.ceph.com/teuthology/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953941/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea7e44db450c4b6292e3cbea8c9ba334 Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) [6953945] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:12:13 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953945/ log: http://qa-proxy.ceph.com/teuthology/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953945/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bfd5ef4d91ce4449b50e09d0db45ecc2 Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) [6953946] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:39:46 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953946/ log: http://qa-proxy.ceph.com/teuthology/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953946/ "2022-07-29T23:37:28.025217+0000 mon.a (mon.0) 3196 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log Pass ================================================================= [6953935] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:26:26 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953935/ [6953936] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} time: 00:49:51 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953936/ [6953938] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:30:22 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953938/ [6953939] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:21:56 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953939/ [6953940] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} time: 00:54:37 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953940/ [6953942] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:28:52 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953942/ [6953943] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:33:54 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953943/ [6953944] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} time: 00:51:03 info: http://pulpito.ceph.com/rraja-2022-07-29_22:43:34-fs:volumes-wip-khiremat-46599-fip-md5-mgr-volumes-distro-default-smithi/6953944/