2022-06-24T04:36:46.718 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-24T04:36:46.720 INFO:teuthology.results:subset: '1/7' 2022-06-24T04:36:46.720 INFO:teuthology.results:seed: '2362' 2022-06-24T05:43:52.348 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-24T05:43:52.390 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-06-24T17:34:54.255 INFO:teuthology.results:Tests finished! gathering results... 2022-06-24T17:35:01.804 INFO:scrape:Found 90 jobs 2022-06-24T17:35:08.522 INFO:scrape:Found 6 distinct failure reasons 2022-06-24T17:35:08.525 INFO:scrape:Failure: reached maximum tries (120) after waiting for 120 seconds 2022-06-24T17:35:08.525 INFO:scrape:24 jobs: ['6895952', '6895968', '6896023', '6895981', '6895978', '6896008', '6895966', '6895971', '6895996', '6896013', '6896018', '6895986', '6896019', '6896038', '6896033', '6896028', '6896030', '6895976', '6896006', '6895987', '6895992', '6896003', '6895962', '6895953'] 2022-06-24T17:35:08.527 INFO:scrape:suites intersection: ['1-start'] 2022-06-24T17:35:08.528 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/16.2.5', '1-start', '2-nfs', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-services/basic', '2-services/client-keyring', '2-services/iscsi', '2-services/mirror', '2-services/nfs', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/nfs2', '2-services/rgw', '2-services/rgw-ingress', '3-final}', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/osds/{0-distro/ubuntu_20.04', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04'] 2022-06-24T17:35:08.529 INFO:scrape: 2022-06-24T17:35:08.530 INFO:scrape:Failure: Command failed on smithi080 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06fb219c-f37c-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi080:/dev/nvme4n1' 2022-06-24T17:35:08.530 INFO:scrape:8 jobs: ['6895980', '6895991', '6896002', '6896012', '6895970', '6896022', '6896032', '6895961'] 2022-06-24T17:35:08.531 INFO:scrape:suites intersection: ['0-nvme-loop', 'fixed-2', 'start}'] 2022-06-24T17:35:08.532 INFO:scrape:suites union: ['0-nvme-loop', 'agent/off', 'agent/on', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'orch:cephadm/smoke/{0-distro/ubuntu_20.04', 'start}'] 2022-06-24T17:35:08.533 INFO:scrape: 2022-06-24T17:35:08.534 INFO:scrape:Dead: b'2022-06-24T17:12:27.111 DEBUG:teuthology.exit:Finished running handlers' 2022-06-24T17:35:08.535 INFO:scrape:3 jobs: ['6895990', '6895955', '6896017'] 2022-06-24T17:35:08.536 INFO:scrape:suites intersection: ['1-start'] 2022-06-24T17:35:08.537 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '2-ops/repave-all}', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8'] 2022-06-24T17:35:08.537 INFO:scrape: 2022-06-24T17:35:08.538 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi164 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f4302d57b213bd3096398c95e2dd8de1620dc6a8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2022-06-24T17:35:08.539 INFO:scrape:2 jobs: ['6895979', '6896031'] 2022-06-24T17:35:08.540 INFO:scrape:suites intersection: ['orch:cephadm/workunits/{agent/off', 'task/test_cephadm}'] 2022-06-24T17:35:08.541 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'orch:cephadm/workunits/{agent/off', 'task/test_cephadm}'] 2022-06-24T17:35:08.541 INFO:scrape: 2022-06-24T17:35:08.542 INFO:scrape:Failure: timeout expired in wait_until_healthy 2022-06-24T17:35:08.542 INFO:scrape:2 jobs: ['6895965', '6896007'] 2022-06-24T17:35:08.543 INFO:scrape:suites intersection: ['mon_election/classic', 'orch:cephadm/workunits/{agent/on'] 2022-06-24T17:35:08.543 INFO:scrape:suites union: ['mon_election/classic', 'orch:cephadm/workunits/{agent/on', 'task/test_nfs}', 'task/test_orch_cli}'] 2022-06-24T17:35:08.544 INFO:scrape: 2022-06-24T17:35:08.545 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) 2022-06-24T17:35:08.545 INFO:scrape:1 jobs: ['6895951'] 2022-06-24T17:35:08.546 INFO:scrape:suites: ['mon_election/connectivity', 'orch:cephadm/workunits/{agent/off', 'task/test_nfs}'] 2022-06-24T17:35:08.546 INFO:scrape: 2022-06-24T17:35:08.549 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/ failed: 37 dead: 3 running: 0 waiting: 0 queued: 0 passed: 50 Fail ================================================================= [6895951] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:27:52 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895951/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895951/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2ab83ed36d024ad2a71bbbd4d76bb862 Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [6895952] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:14:04 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895952/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895952/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0022b57c435e43fa87c4a5a703752c65 reached maximum tries (120) after waiting for 120 seconds [6895953] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:19:22 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895953/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895953/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14f06e079eb544c0a848a81ee0203f49 reached maximum tries (120) after waiting for 120 seconds [6895961] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:08:58 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895961/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895961/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed06a86d236f4884929213cd3fc25416 Command failed on smithi160 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid da8cbb90-f379-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi160:/dev/nvme4n1' [6895962] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:17:19 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895962/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895962/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cc3752cbd12b4725b672f6e238372b53 reached maximum tries (120) after waiting for 120 seconds [6895965] orch:cephadm/workunits/{agent/on mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:19:09 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895965/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f83efa92eb5a49e8a14afd83e869f507 timeout expired in wait_until_healthy [6895966] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:28:35 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895966/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895966/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b466b9130f9a42a190e37caf5120f6ad reached maximum tries (120) after waiting for 120 seconds [6895968] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:19:02 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895968/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895968/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b693e3daa30b425796393255f4316e87 reached maximum tries (120) after waiting for 120 seconds [6895970] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895970/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895970/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2300ee7c0d7844aca42c8eda8a79b77d Command failed on smithi181 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f22fbe14-f379-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi181:/dev/nvme4n1' [6895971] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:13:19 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895971/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895971/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30d73cc6ee684053b49ecfcfaa13a3dd reached maximum tries (120) after waiting for 120 seconds [6895976] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:13:36 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895976/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895976/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dacd6146d1d345968d025f4664ea1834 reached maximum tries (120) after waiting for 120 seconds [6895978] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:17:05 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895978/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895978/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d944500c21b1495481d96273391a6184 reached maximum tries (120) after waiting for 120 seconds [6895979] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:14:32 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895979/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895979/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5d481b3a38c45c098b1e7d51dc81eda Command failed (workunit test cephadm/test_cephadm.sh) on smithi164 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f4302d57b213bd3096398c95e2dd8de1620dc6a8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [6895980] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:32 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895980/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895980/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a44b44be3ded4089b009ac137fc48f8c Command failed on smithi080 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 06fb219c-f37c-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi080:/dev/nvme4n1' [6895981] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} ----------------------------------------------------------------- time: 00:19:24 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895981/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895981/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e3b49353ed844d6a6d6ad1152c7e241 reached maximum tries (120) after waiting for 120 seconds [6895986] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:42:22 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895986/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895986/ sentry: https://sentry.ceph.com/organizations/ceph/?query=08132e8282894306850c57129065a426 reached maximum tries (300) after waiting for 300 seconds [6895987] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:17:57 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895987/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895987/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec42855dfd6645a0a276111e18c06f90 reached maximum tries (120) after waiting for 120 seconds [6895991] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:12:17 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895991/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895991/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3fc29a52a6234839982b509b4b996c92 Command failed on smithi019 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3e2f1906-f37d-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi019:/dev/nvme4n1' [6895992] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:27:16 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895992/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895992/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6bb17f1ba9114777927caeb5ebf318fb reached maximum tries (120) after waiting for 120 seconds [6895996] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:13:57 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895996/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895996/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d967845b5fb64780bbf920dde2b90c9f reached maximum tries (120) after waiting for 120 seconds [6896002] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:10:24 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896002/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896002/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5d84e108e7824c25a3c341e1ba356e23 Command failed on smithi043 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12b03a0c-f37e-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi043:/dev/nvme4n1' [6896003] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final} ----------------------------------------------------------------- time: 00:14:08 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896003/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896003/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6f3f9fa9ed46406ea4c5dfb67347c13e reached maximum tries (120) after waiting for 120 seconds [6896006] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:27:17 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896006/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896006/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d632b2c06f6a4da6a984c53dbae99bef reached maximum tries (120) after waiting for 120 seconds [6896007] orch:cephadm/workunits/{agent/on mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:19:03 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896007/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896007/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3da82d0751c241e2bf087f10a57aa1b4 timeout expired in wait_until_healthy [6896008] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:20:49 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896008/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896008/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ab1a48f445a4a4f8f149cefaedad65a reached maximum tries (120) after waiting for 120 seconds [6896012] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:08:57 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896012/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896012/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30e1ad13b0d147ca85b708b6ea8779da Command failed on smithi036 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73997bbc-f37e-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi036:/dev/nvme4n1' [6896013] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:17:21 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896013/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896013/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ff1938857ab404fa7294535a44d4062 reached maximum tries (120) after waiting for 120 seconds [6896018] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:13:46 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896018/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896018/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0de87e30b5e64e499ad16657e965163f reached maximum tries (120) after waiting for 120 seconds [6896019] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:28:00 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896019/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896019/ sentry: https://sentry.ceph.com/organizations/ceph/?query=798adf52cd824f0b8dabe115ef45dcdd reached maximum tries (120) after waiting for 120 seconds [6896022] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:06:17 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896022/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896022/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6882dd78f264a43a3d0c5d9003c2546 Command failed on smithi018 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f4db0384-f37f-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi018:/dev/nvme4n1' [6896023] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:13:24 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896023/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896023/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12c938f573f846f694f87b8530d711d3 reached maximum tries (120) after waiting for 120 seconds [6896028] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:13:34 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896028/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896028/ sentry: https://sentry.ceph.com/organizations/ceph/?query=77b3194901cb4bd7bad0d300c6940d2d reached maximum tries (120) after waiting for 120 seconds [6896030] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:12:43 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896030/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896030/ sentry: https://sentry.ceph.com/organizations/ceph/?query=659bf328264f4feea5a5a9f6291397f7 reached maximum tries (120) after waiting for 120 seconds [6896031] orch:cephadm/workunits/{agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:14:09 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896031/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896031/ sentry: https://sentry.ceph.com/organizations/ceph/?query=487edd871e8a4a019fb4f967235be2cb Command failed (workunit test cephadm/test_cephadm.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f4302d57b213bd3096398c95e2dd8de1620dc6a8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [6896032] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896032/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896032/ sentry: https://sentry.ceph.com/organizations/ceph/?query=68cb63601cb947098787dc0c101e1ea2 Command failed on smithi036 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:f4302d57b213bd3096398c95e2dd8de1620dc6a8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8383378-f380-11ec-842b-001a4aab830c -- ceph orch daemon add osd smithi036:/dev/nvme4n1' [6896033] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:18:26 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896033/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896033/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a080a0f350ee4615ae114c550ff7b697 reached maximum tries (120) after waiting for 120 seconds [6896038] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:18:05 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896038/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896038/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03279bf6f0ac4dfca0c945a01fe92011 reached maximum tries (120) after waiting for 120 seconds Dead ================================================================= [6895955] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895955/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895955/ hit max job timeout [6895990] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895990/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895990/ hit max job timeout [6896017] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896017/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896017/ hit max job timeout Pass ================================================================= [6895950] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:17 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895950/ [6895954] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:26:42 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895954/ [6895956] orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} time: 00:11:39 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895956/ [6895957] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:30:43 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895957/ [6895958] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:12:29 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895958/ [6895959] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:15 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895959/ [6895960] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:52:45 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895960/ [6895963] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:57:44 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895963/ [6895964] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:38:04 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895964/ [6895967] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:38 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895967/ [6895969] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:15 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895969/ [6895972] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:29:18 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895972/ [6895973] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:33:02 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895973/ [6895974] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:33:14 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895974/ [6895975] orch:cephadm/workunits/{agent/on mon_election/classic task/test_adoption} time: 00:08:09 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895975/ [6895977] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:20:38 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895977/ [6895982] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:36:34 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895982/ [6895983] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:40:24 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895983/ [6895984] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:21 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895984/ [6895985] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:43:04 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895985/ [6895988] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:27:09 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895988/ [6895989] orch:cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos} time: 00:03:54 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895989/ [6895993] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:40:40 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895993/ [6895994] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:42:02 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895994/ [6895995] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:26 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895995/ [6895997] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:30:58 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895997/ [6895998] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:20:47 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895998/ [6895999] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:11:35 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6895999/ [6896000] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/on orchestrator_cli} time: 00:12:59 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896000/ [6896001] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:09:58 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896001/ [6896004] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:52:51 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896004/ [6896005] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:32:49 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896005/ [6896009] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:28:32 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896009/ [6896010] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:13 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896010/ [6896011] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:53:11 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896011/ [6896014] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:41:12 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896014/ [6896015] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:37:39 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896015/ [6896016] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_orch_cli} time: 00:16:25 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896016/ [6896020] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:51 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896020/ [6896021] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:56 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896021/ [6896024] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:32:46 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896024/ [6896025] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:41:45 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896025/ [6896026] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:33:05 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896026/ [6896027] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_adoption} time: 00:07:09 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896027/ [6896029] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:20:08 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896029/ [6896034] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:32:01 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896034/ [6896035] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:38:51 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896035/ [6896036] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:28 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896036/ [6896037] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:31:33 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896037/ [6896039] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:25:31 info: http://pulpito.ceph.com/adking-2022-06-24_01:35:59-orch:cephadm-wip-adk-testing-2022-06-23-1830-distro-default-smithi/6896039/