2023-12-11T13:22:26.300 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-11T13:22:26.300 INFO:teuthology.results:subset: '1/14' 2023-12-11T13:22:26.301 INFO:teuthology.results:seed: '6285' 2023-12-11T13:23:23.697 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-11T13:23:23.701 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-12-12T01:24:19.719 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-12-12T01:24:21.116 INFO:scrape:Found 16 jobs 2023-12-12T01:24:24.438 INFO:scrape:Found 6 distinct failure reasons 2023-12-12T01:24:24.438 INFO:scrape:Timeout 3h running clone.client.1/qa/workunits/suites/fsstress.sh 2023-12-12T01:24:24.438 INFO:scrape:2 jobs: ['7486917', '7486911'] 2023-12-12T01:24:24.440 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2023-12-12T01:24:24.440 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2023-12-12T01:24:24.440 INFO:scrape: 2023-12-12T01:24:24.440 INFO:scrape:Failure: SSH connection to smithi149 was lost: 'sudo yum -y install ceph-radosgw' 2023-12-12T01:24:24.440 INFO:scrape:1 jobs: ['7486916'] 2023-12-12T01:24:24.441 INFO:scrape:suites: ['agent/off', 'mon_election/classic', 'orch:cephadm/workunits/{0-distro/centos_9.stream_runc', 'task/test_iscsi_container/{centos_9.stream', 'test_iscsi_container}}'] 2023-12-12T01:24:24.441 INFO:scrape: 2023-12-12T01:24:24.441 INFO:scrape:Failure: Command failed (workunit test cephadm/test_repos.sh) on smithi114 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7f6a7c6a73aa26830ad10d37928d11cbae0148f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' 2023-12-12T01:24:24.441 INFO:scrape:2 jobs: ['7486913', '7486922'] 2023-12-12T01:24:24.441 INFO:scrape:suites intersection: ['agent/on', 'task/test_cephadm_repos}'] 2023-12-12T01:24:24.441 INFO:scrape:suites union: ['agent/on', 'mon_election/classic', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/centos_9.stream', 'orch:cephadm/workunits/{0-distro/centos_9.stream_runc', 'task/test_cephadm_repos}'] 2023-12-12T01:24:24.441 INFO:scrape: 2023-12-12T01:24:24.441 INFO:scrape:Failure: Command failed on smithi027 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc74174e-982a-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2023-12-12T01:24:24.441 INFO:scrape:6 jobs: ['7486908', '7486918', '7486909', '7486910', '7486919', '7486912'] 2023-12-12T01:24:24.441 INFO:scrape:suites intersection: [] 2023-12-12T01:24:24.442 INFO:scrape:suites union: ['1-bootstrap/17.2.0', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/no', '3-upgrade-mgr-staggered', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-config-upgrade/{fail_fs}', '4-final}', '4-verify}', '4-wait', '5-upgrade-ls', '5-upgrade-with-workload', '6-verify}}', 'agent/off', 'agent/on', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/classic', 'mon_election/classic}', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream', 'orch:cephadm/workunits/{0-distro/centos_9.stream', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'task/test_rgw_multisite}', 'tasks/{0-from/quincy'] 2023-12-12T01:24:24.442 INFO:scrape: 2023-12-12T01:24:24.442 INFO:scrape:Failure: ['/dev/vg_nvme: already exists in filesystem', "Creating volume group 'vg_nvme' failed", "Run `vgcreate --help' for more information."] 2023-12-12T01:24:24.442 INFO:scrape:1 jobs: ['7486915'] 2023-12-12T01:24:24.442 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2023-12-12T01:24:24.442 INFO:scrape: 2023-12-12T01:24:24.442 INFO:scrape:Dead: b'2023-12-12T01:10:26.216 INFO:journalctl@ceph.mon.smithi173.smithi173.stdout:Dec 12 01:08:24 smithi173 ceph-e83eff30-982a-11ee-95a3-87774f69a715-mon-smithi173[89212]: 2023-12-12T01:08:23.571+0000 7f4323402700 -1 mon.smithi173@1(probing) e2 get_health_metrics reporting 335180 slow ops, oldest is log(1 entries from seq 47 at 2023-12-11T13:47:54.192208+0000)' 2023-12-12T01:24:24.442 INFO:scrape:1 jobs: ['7486923'] 2023-12-12T01:24:24.443 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2023-12-12T01:24:24.443 INFO:scrape: 2023-12-12T01:24:24.444 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/ failed: 11 dead: 1 running: 1 waiting: 0 queued: 0 passed: 3 Fail ================================================================= [7486908] orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:24:23 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486908/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486908/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f93a41c5c72b48d291daa3fd3d171f18 Command failed on smithi027 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc74174e-982a-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7486909] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:25:00 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486909/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486909/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c79ebc226939487faee7b5d301fe5924 Command failed on smithi141 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6dfc5f78-982b-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7486910] orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} ----------------------------------------------------------------- time: 00:13:24 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486910/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9897d00a3cbb4de187de204594704794 Command failed on smithi022 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:a7f6a7c6a73aa26830ad10d37928d11cbae0148f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 499eaf64-982b-11ee-95a3-87774f69a715 -- ceph rgw realm bootstrap -i -' [7486911] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 03:34:45 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486911/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486911/ sentry: https://sentry.ceph.com/organizations/ceph/?query=79663bd6b0cd48c78222dc8576a894c6 Command failed (workunit test suites/fsstress.sh) on smithi160 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7f6a7c6a73aa26830ad10d37928d11cbae0148f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh' [7486912] orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:40:50 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486912/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486912/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df26c888bc5d4f168b1203d4816e6e9b Command failed on smithi031 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid da04d840-982a-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.foo'" [7486913] orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:07:05 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486913/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486913/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3766ed81241c4125b53b6dfa09e6cb2e Command failed (workunit test cephadm/test_repos.sh) on smithi114 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7f6a7c6a73aa26830ad10d37928d11cbae0148f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7486916] orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} ----------------------------------------------------------------- time: 00:08:36 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486916/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486916/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9f6f46df9fb487b86a1479ba3c65d70 SSH connection to smithi149 was lost: 'sudo yum -y install ceph-radosgw' [7486917] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 03:35:03 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486917/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486917/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fdbe3316f11b4451b173c2f596347726 Command failed (workunit test suites/fsstress.sh) on smithi018 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7f6a7c6a73aa26830ad10d37928d11cbae0148f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh' [7486918] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:27:25 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486918/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486918/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30bd333524a24d47bf91ad7e650d7dd9 Command failed on smithi125 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1cb3adc-982a-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7486919] orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:23:42 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486919/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486919/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b8e40f68d7414348aed2379f69650703 Command failed on smithi033 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e377592-982a-11ee-95a3-87774f69a715 -e sha1=a7f6a7c6a73aa26830ad10d37928d11cbae0148f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7486922] orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:06:56 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486922/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486922/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fa9e683efe4d46178935daa1cfb2f6a6 Command failed (workunit test cephadm/test_repos.sh) on smithi169 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a7f6a7c6a73aa26830ad10d37928d11cbae0148f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' Dead ================================================================= [7486915] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:05:45 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486915/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486915/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44dcfb6b0ec64c3a805b5fa21de54a1d ['/dev/vg_nvme: already exists in filesystem', "Creating volume group 'vg_nvme' failed", "Run `vgcreate --help' for more information."] Running ================================================================= [7486923] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486923/ Pass ================================================================= [7486914] orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} time: 00:12:16 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486914/ [7486920] orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_ca_signed_key} time: 00:12:07 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486920/ [7486921] orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:23:12 info: https://pulpito.ceph.com/adking-2023-12-11_13:22:20-orch:cephadm-wip-adk4-centos-9-runc-2023-12-10-1421-distro-default-smithi/7486921/