2022-11-20T20:50:13.621 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-20T20:50:13.621 INFO:teuthology.results:seed: '7840' 2022-11-20T20:50:55.205 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-20T20:50:55.206 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-11-20T21:25:57.585 INFO:teuthology.results:Tests finished! gathering results... 2022-11-20T21:25:58.000 INFO:scrape:Found 10 jobs 2022-11-20T21:25:58.837 INFO:scrape:Found 2 distinct failure reasons 2022-11-20T21:25:58.838 INFO:scrape:Failure: Command failed on smithi121 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a09dd4c-6917-11ed-8438-001a4aab830c -e sha1=910cff70b59f29e584f743f9b020c9d5056f86a3 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' 2022-11-20T21:25:58.838 INFO:scrape:1 jobs: ['7086038'] 2022-11-20T21:25:58.839 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/v16.2.4'] 2022-11-20T21:25:58.839 INFO:scrape: 2022-11-20T21:25:58.839 INFO:scrape:Failure: Command failed on smithi090 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' 2022-11-20T21:25:58.839 INFO:scrape:5 jobs: ['7086037', '7086039', '7086035', '7086031', '7086034'] 2022-11-20T21:25:58.840 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/pacific'] 2022-11-20T21:25:58.840 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/pacific'] 2022-11-20T21:25:58.840 INFO:scrape: 2022-11-20T21:25:58.841 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/ failed: 6 dead: 0 running: 0 waiting: 0 queued: 0 passed: 4 Fail ================================================================= [7086031] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:10:33 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086031/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086031/ sentry: https://sentry.ceph.com/organizations/ceph/?query=11eec5e04a5845a3b0399685bac00772 Command failed on smithi008 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' [7086034] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:10:16 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086034/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086034/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b371dcf0a1244c6bbdae4e0bddaa66b8 Command failed on smithi055 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' [7086035] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:10:18 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086035/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086035/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aacbbee641114dfe994aa31f81f8120f Command failed on smithi138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' [7086037] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:09:59 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086037/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086037/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7d80de265fcf44cd96c4eac1e0327260 Command failed on smithi090 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' [7086038] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:22:14 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086038/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086038/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d8d6ffb5afd947d69d11ba4a47b81838 Command failed on smithi121 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a09dd4c-6917-11ed-8438-001a4aab830c -e sha1=910cff70b59f29e584f743f9b020c9d5056f86a3 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' [7086039] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:10:09 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086039/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086039/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7bb13d366d3c44e1b9736dce88648e5a Command failed on smithi116 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' Pass ================================================================= [7086030] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:29:01 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086030/ [7086032] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} time: 00:12:29 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086032/ [7086033] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:12:51 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086033/ [7086036] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:20:06 info: http://pulpito.ceph.com/trociny-2022-11-20_20:49:26-fs-wip-mgolub-testing-distro-default-smithi/7086036/