2022-10-08T00:10:29.413 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-08T00:10:29.415 INFO:teuthology.results:subset: '1/8' 2022-10-08T00:10:29.415 INFO:teuthology.results:seed: '1922' 2022-10-08T00:45:10.933 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-08T00:45:10.960 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-10-08T01:22:13.491 INFO:teuthology.results:Tests finished! gathering results... 2022-10-08T01:22:16.184 INFO:scrape:Found 97 jobs 2022-10-08T01:22:18.063 INFO:scrape:Found 3 distinct failure reasons 2022-10-08T01:22:18.064 INFO:scrape:Failure: Command failed on smithi109 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bee08d70-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' 2022-10-08T01:22:18.064 INFO:scrape:55 jobs 2022-10-08T01:22:18.065 INFO:scrape:suites intersection: [] 2022-10-08T01:22:18.065 INFO:scrape:suites union: ['0-nvme-loop', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-repo_digest/defaut', '2-services/basic', '2-services/client-keyring', '2-services/mirror', '2-services/rgw', '2-thrash', '3-final}', '3-inline/yes', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', '3-upgrade-mgr-staggered', '3-upgrade/staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '4-wait', '5-upgrade-ls', '5-upgrade-with-workload', '6-verify}}', 'agent/off', 'agent/on', 'base/install', 'centos_8.stream_container_tools', 'cluster/{fixed-3', 'conf/{client', 'fail_fs/yes', 'fixed-2', 'ignorelist_wrongly_marked_down', 'mds', 'mode/packaged', 'mode/root', 'mon', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'openstack}', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/rbd_iscsi/{0-single-container-host', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools}', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/smoke/{0-distro/ubuntu_20.04', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/thrash/{0-distro/ubuntu_20.04', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/with-work/{0-distro/ubuntu_20.04', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/workunits/{0-distro/ubuntu_20.04', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'root}', 'start', 'start}', 'syntax}', 'task/test_iscsi_pids_limit/{centos_8.stream_container_tools', 'task/test_nfs}', 'task/test_orch_cli_mon}', 'task/test_orch_cli}', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'tasks/rotate-keys}', 'tasks/{0-from/pacific', 'test_iscsi_pids_limit}}', 'workloads/cephadm_iscsi}'] 2022-10-08T01:22:18.066 INFO:scrape: 2022-10-08T01:22:18.066 INFO:scrape:Failure: Command failed on smithi176 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' 2022-10-08T01:22:18.067 INFO:scrape:1 jobs: ['7058161'] 2022-10-08T01:22:18.067 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'task/test_iscsi_pids_limit/{centos_8.stream_container_tools', 'test_iscsi_pids_limit}}'] 2022-10-08T01:22:18.068 INFO:scrape: 2022-10-08T01:22:18.068 INFO:scrape:Failure: Command failed on smithi109 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935d17e2-46a3-11ed-8436-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-10-08T01:22:18.068 INFO:scrape:3 jobs: ['7058241', '7058193', '7058160'] 2022-10-08T01:22:18.069 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start'] 2022-10-08T01:22:18.069 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', '2-ops/rm-zap-wait}', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/osds/{0-distro/ubuntu_20.04'] 2022-10-08T01:22:18.070 INFO:scrape: 2022-10-08T01:22:18.073 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/ failed: 59 dead: 0 running: 0 waiting: 0 queued: 0 passed: 38 Fail ================================================================= [7058160] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:14:50 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058160/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058160/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8530ebb9daf4fdd815e0d55a98e35bc Command failed on smithi085 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b96a661e-469f-11ed-8436-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7058161] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} ----------------------------------------------------------------- time: 00:04:16 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058161/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058161/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f620c39d8f6d405fab73792e22d0c6e3 Command failed on smithi176 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' [7058162] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:20:59 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058162/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058162/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14c56c7114414d6e8c4dadbf9bfc152d Command failed on smithi149 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69f72926-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058167] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} ----------------------------------------------------------------- time: 00:11:17 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058167/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058167/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03d8e275cccc42849434f418b1def1f3 Command failed on smithi040 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 251f70fc-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058168] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:07:40 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058168/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058168/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3bd9e3a1cb2849dcb20235cbf0697bdd Command failed on smithi111 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 558d5084-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058169] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:43:11 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058169/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058169/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e19de4f73d449adb8ec7369e5fc2630 Command failed on smithi007 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 16f39562-46a0-11ed-8436-001a4aab830c -e sha1=32a5ea00c4bee0c51c66aa8902716d7900f94c76 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.r'" [7058170] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:23:11 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058170/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058170/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9a5f1786ad743bf944d449ae65f2179 Command failed on smithi063 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd0e69f2-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058171] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:19:33 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058171/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058171/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a74031466d044f8a819cc22acd718f01 Command failed on smithi027 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680666d6-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058173] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:19:08 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058173/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058173/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c021a595a754ed68711cea6c4b69f06 Command failed on smithi037 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7d008504-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058174] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:23:10 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058174/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058174/ sentry: https://sentry.ceph.com/organizations/ceph/?query=629e51c9668e433db2ea926e35821bef Command failed on smithi018 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce2c6902-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058175] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:17:59 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058175/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058175/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c91fe58800f041d8b2e354123eab920f Command failed on smithi049 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99d320d8-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058176] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:12:35 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058176/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058176/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd544f6b73a54d2e87097f338747ce43 Command failed on smithi035 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 047d0a62-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058178] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:16:49 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058178/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058178/ sentry: https://sentry.ceph.com/organizations/ceph/?query=87bf7b9eaad649788013c24b67dcc399 Command failed on smithi012 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97db7500-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058180] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:16:30 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058180/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058180/ sentry: https://sentry.ceph.com/organizations/ceph/?query=498a18187986445b8c183ad70d0e945b Command failed on smithi172 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85add710-46a0-11ed-8436-001a4aab830c -- ceph orch daemon add osd smithi172:vg_nvme/lv_4' [7058181] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:20:51 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058181/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058181/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ef478f34b2d407eb928e171d43f96de Command failed on smithi033 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c317dc72-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058182] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:12:58 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058182/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058182/ sentry: https://sentry.ceph.com/organizations/ceph/?query=318ec972f58747a8b20e073176218070 Command failed on smithi005 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9d44912-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058184] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:11:18 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058184/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058184/ sentry: https://sentry.ceph.com/organizations/ceph/?query=86e3c8becbc442b881b0b48f40ef0b5c Command failed on smithi044 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd749f66-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058185] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:16:26 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058185/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058185/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d7c41eab939b49f58ee69f3a80301e3e Command failed on smithi109 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bee08d70-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058186] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:23:44 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058186/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058186/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9cb62ee34d9841cc9c1f81527eed8f6f Command failed on smithi186 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7a01270-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058189] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} ----------------------------------------------------------------- time: 00:22:57 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058189/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058189/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1fff7e6ef5ba4f3d9b25f49c6ea19f2c Command failed on smithi134 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f714098-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058193] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:22:39 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058193/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058193/ sentry: https://sentry.ceph.com/organizations/ceph/?query=74e9e0e958c5436aa4c389c7806be418 Command failed on smithi080 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5e0ce052-46a0-11ed-8436-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7058194] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:17:03 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058194/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058194/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ba9d9e7c5754dcbb8455c803d945775 Command failed on smithi150 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf5f5b0a-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058198] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:09:34 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058198/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058198/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b69146b01c2545d8bd5435c255b0ae1d Command failed on smithi059 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c6168d8-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058199] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:16:12 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058199/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058199/ sentry: https://sentry.ceph.com/organizations/ceph/?query=382a2466236f4307aedd1e0c400d4f50 Command failed on smithi081 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a400cf10-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058200] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:15:58 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058200/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058200/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16a5a5b95b99470ea74b65b2a13d38b0 Command failed on smithi046 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 52370fd2-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058202] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} ----------------------------------------------------------------- time: 00:11:46 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058202/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058202/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0dd9f4196a53436baa0aad889b352542 Command failed on smithi138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c5031fe8-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058204] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:16:10 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058204/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058204/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea10eeaa45614b1394f83f79e6dc45fb Command failed on smithi171 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4bd10c88-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058206] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:23:39 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058206/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058206/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a03efada16094ee485b3c26c22a0a8f3 Command failed on smithi120 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c024358-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058207] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:11:33 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058207/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058207/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee37e19c37f145ac9232479036d51059 Command failed on smithi096 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cf431454-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058209] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:10:46 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058209/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058209/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5cfbb5aa18084299841809283dc8650c Command failed on smithi032 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8081a50-469f-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058210] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:15:58 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058210/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058210/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1534b4d46d564b91a15d4ed92da1c983 Command failed on smithi099 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4ae18b86-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058211] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:20:55 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058211/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058211/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7cbe22bbe1647838c077c07b66c9b08 Command failed on smithi097 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6d149580-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058213] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:14:37 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058213/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058213/ sentry: https://sentry.ceph.com/organizations/ceph/?query=af9025d5e4d744e8a052edc1b454033b Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70032b36-46a0-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058216] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:05:52 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058216/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058216/ sentry: https://sentry.ceph.com/organizations/ceph/?query=26c584ed6f524e8bb437119d3e189339 Command failed on smithi153 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95e0cfe2-46a1-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058217] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:39:45 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058217/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058217/ sentry: https://sentry.ceph.com/organizations/ceph/?query=069a12a88ab948a1b88659c899a7b57c Command failed on smithi032 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a51f9006-46a1-11ed-8436-001a4aab830c -e sha1=32a5ea00c4bee0c51c66aa8902716d7900f94c76 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.r'" [7058218] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:11:16 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058218/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058218/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d702a5d97d654cd6af08c14a7f79feb2 Command failed on smithi096 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9dd2c6a-46a2-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058220] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:19:02 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058220/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058220/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a22f59d548840a2a09adacdaf31e416 Command failed on smithi040 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98dd0bc8-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058221] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:10:29 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058221/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058221/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8673a8aa721d4ec8a1a55bb3dd44139e Command failed on smithi084 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aed08514-46a2-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058223] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:12:01 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058223/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058223/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e8f36b4b6be4770a3673c12578a1abf Command failed on smithi103 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23fe4e34-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058225] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:17:22 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058225/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058225/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bda2e75dbe1a4bb8b5671ab0dfd812f7 Command failed on smithi171 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ef9929f6-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058226] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:11:09 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058226/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058226/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2f6b5cf49804f36a0037e4fd58accea Command failed on smithi046 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2597dd1e-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058229] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:17:17 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058229/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058229/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f84448a1eed24dd5bddfc2d715179272 Command failed on smithi037 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2296ad74-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058231] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:14:11 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058231/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058231/ sentry: https://sentry.ceph.com/organizations/ceph/?query=51bb1f37843d424bb7c525fd1eafab27 Command failed on smithi036 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80cf5752-46a3-11ed-8436-001a4aab830c -- ceph osd stat -f json' [7058233] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:16:50 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058233/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058233/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c9a533314d1430a8fec940ecdeec965 Command failed on smithi012 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66ef3428-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058236] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:12:23 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058236/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058236/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0a62eafca57d4c18a684e3c51d4a165e Command failed on smithi018 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c077dea6-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058237] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:16:56 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058237/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058237/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f01565d9e26f474da200b0b4c626159a Command failed on smithi063 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 78811300-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058238] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:12:12 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058238/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058238/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fcacd773f2c740ed848fc217445b04bf Command failed on smithi090 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39120238-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058241] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:14:51 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058241/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058241/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e93efcc6372840fea7097a09267e2a34 Command failed on smithi109 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935d17e2-46a3-11ed-8436-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7058242] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} ----------------------------------------------------------------- time: 00:10:33 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058242/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058242/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c422772dbf3b4d8c8a53dfb6222e9fe4 Command failed on smithi027 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f9acf6f2-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058243] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:11:39 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058243/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058243/ sentry: https://sentry.ceph.com/organizations/ceph/?query=26e604f725ff4c98b49c305b4e05acb6 Command failed on smithi134 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f3c400b4-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058245] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:11:38 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058245/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058245/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e017514359d74f6aa5327416dd3d0e04 Command failed on smithi043 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0cccdf04-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058246] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:16:26 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058246/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058246/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8159a663eeb54093b2f46067b744f30e Command failed on smithi033 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b14cacc6-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058247] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:08:54 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058247/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058247/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6fbc108d53a48de936e5fab9e7c17fc Command failed on smithi049 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 801bb2c4-46a3-11ed-8436-001a4aab830c -- ceph osd stat -f json' [7058248] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:06:52 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058248/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058248/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e98468b44af442ff97a2d01f8b21415f Command failed on smithi186 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9ff248a-46a3-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/nvme4n1' [7058249] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:11:53 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058249/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058249/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bc9e3f0ef294922b6eec0bab37ee266 Command failed on smithi153 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7d51abba-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058250] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:16:38 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058250/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058250/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a4f12bc4d4504ac6b1d23fb70864392e Command failed on smithi097 with status 1: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce6ee44a-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058252] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:16:12 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058252/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058252/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b41e3f7a93134899bb3c791a8b85bb33 Command failed on smithi137 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f33d11a2-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058254] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:16:33 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058254/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058254/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5196e93c9d0f48c4858396cfa9c517b3 Command failed on smithi120 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fcb4fa9c-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' [7058256] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:15:10 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058256/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058256/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b8815c5b224e4336aacfd53df3576a63 Command failed on smithi045 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:32a5ea00c4bee0c51c66aa8902716d7900f94c76 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d19df28c-46a4-11ed-8436-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4' Pass ================================================================= [7058163] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} time: 00:15:51 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058163/ [7058164] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:35:40 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058164/ [7058165] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:40:37 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058165/ [7058166] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/off orchestrator_cli} time: 00:16:51 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058166/ [7058172] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:22:36 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058172/ [7058177] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:26:19 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058177/ [7058179] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:58 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058179/ [7058183] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:20:14 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058183/ [7058187] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_adoption} time: 00:12:53 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058187/ [7058188] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:17:53 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058188/ [7058190] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:35:15 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058190/ [7058191] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:35:14 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058191/ [7058192] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_cephadm} time: 00:27:15 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058192/ [7058195] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:16 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058195/ [7058196] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:12:40 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058196/ [7058197] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:41:09 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058197/ [7058201] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:24:25 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058201/ [7058203] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:35:07 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058203/ [7058205] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:24:24 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058205/ [7058208] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:24:05 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058208/ [7058212] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:26:09 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058212/ [7058214] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:34:11 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058214/ [7058215] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/on orchestrator_cli} time: 00:14:35 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058215/ [7058219] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:12:52 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058219/ [7058222] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:35 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058222/ [7058224] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:11:25 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058224/ [7058227] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption} time: 00:16:28 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058227/ [7058228] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:32:00 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058228/ [7058230] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm} time: 00:21:01 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058230/ [7058232] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:12 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058232/ [7058234] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:53 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058234/ [7058235] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:16:26 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058235/ [7058239] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:29:00 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058239/ [7058240] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:28:58 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058240/ [7058244] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/jaeger 3-final} time: 00:18:42 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058244/ [7058251] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:32:57 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058251/ [7058253] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:13:37 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058253/ [7058255] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:11:40 info: http://pulpito.ceph.com/adking-2022-10-08_00:10:21-orch:cephadm-wip-adk-testing-2022-10-07-1318-distro-default-smithi/7058255/