2022-03-18T06:23:34.621 INFO:root:teuthology version: 1.1.0-aa078dbe 2022-03-18T06:23:34.622 INFO:teuthology.results:subset: '1/9' 2022-03-18T06:23:34.622 INFO:teuthology.results:seed: '8426' 2022-03-18T07:16:32.951 INFO:root:teuthology version: 1.1.0-aa078dbe 2022-03-18T07:16:32.966 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-18T13:54:03.295 INFO:teuthology.results:Tests finished! gathering results... 2022-03-18T13:54:09.007 INFO:scrape:Found 80 jobs 2022-03-18T13:54:17.906 INFO:scrape:Found 4 distinct failure reasons 2022-03-18T13:54:17.907 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) 2022-03-18T13:54:17.907 INFO:scrape:1 jobs: ['6743936'] 2022-03-18T13:54:17.908 INFO:scrape:suites: ['mon_election/classic', 'orch:cephadm/workunits/{agent/off', 'task/test_nfs}'] 2022-03-18T13:54:17.908 INFO:scrape: 2022-03-18T13:54:17.909 INFO:scrape:Failure: timeout expired in wait_until_healthy 2022-03-18T13:54:17.909 INFO:scrape:48 jobs 2022-03-18T13:54:17.910 INFO:scrape:suites intersection: [] 2022-03-18T13:54:17.911 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/repave-all}', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-services/basic', '2-services/client-keyring', '2-services/iscsi', '2-services/mirror', '2-services/nfs', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/nfs2', '2-services/rgw', '2-services/rgw-ingress', '2-thrash', '3-final}', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/snaps-few-objects', 'agent/off', 'agent/on', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/osds/{0-distro/ubuntu_20.04', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_3.0}', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8}', 'orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/smoke/{0-distro/ubuntu_20.04', 'orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/thrash/{0-distro/ubuntu_20.04', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0', 'orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8', 'orch:cephadm/with-work/{0-distro/ubuntu_20.04', 'root}', 'start', 'start}', 'tasks/rados_api_tests}', 'tasks/rados_python}'] 2022-03-18T13:54:17.911 INFO:scrape: 2022-03-18T13:54:17.912 INFO:scrape:Failure: Command failed on smithi040 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:93d090eac4a00adb26cea7ff2afbad83805cf2ef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5582a0-a68d-11ec-8c35-001a4aab830c -- ceph orch apply node-exporter '2;smithi025=a;smithi040=b'" 2022-03-18T13:54:17.912 INFO:scrape:1 jobs: ['6743987'] 2022-03-18T13:54:17.912 INFO:scrape:suites: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v2only', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'start', 'tasks/rados_python}'] 2022-03-18T13:54:17.913 INFO:scrape: 2022-03-18T13:54:17.913 INFO:scrape:Dead: b'2022-03-18T13:08:26.344 DEBUG:teuthology.exit:Finished running handlers' 2022-03-18T13:54:17.914 INFO:scrape:14 jobs: ['6743928', '6743974', '6743953', '6743973', '6743941', '6743943', '6743986', '6743959', '6743911', '6743915', '6743940', '6743927', '6743971', '6743985'] 2022-03-18T13:54:17.914 INFO:scrape:suites intersection: [] 2022-03-18T13:54:17.915 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-bootstrap/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/no', '3-inline/yes', '3-start-upgrade', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', '4-wait', '5-upgrade-ls', 'agent/off', 'agent/on', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'mon_election/classic}', 'mon_election/connectivity}', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9', 'osd}', 'overrides/{pg-warn', 'roles', 'syntax', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-18T13:54:17.915 INFO:scrape: 2022-03-18T13:54:17.918 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/ failed: 50 dead: 14 running: 0 waiting: 0 queued: 0 passed: 16 Fail ================================================================= [6743912] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:16:38 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743912/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743912/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b26fde3029ba45dfbb9699188847d0af timeout expired in wait_until_healthy [6743913] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:21:19 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743913/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743913/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cded78331b634d60afe62058befe7d62 timeout expired in wait_until_healthy [6743914] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:35:16 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743914/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743914/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2248add8f9f747299febada7b8600306 timeout expired in wait_until_healthy [6743918] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:15:32 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743918/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743918/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1a79d57d68cb49a7a5127f549794017e timeout expired in wait_until_healthy [6743919] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:20:34 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743919/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743919/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c1fd06208694d6d9ce0f077d9c3d922 timeout expired in wait_until_healthy [6743920] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:23:54 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743920/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743920/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4db14599e5824152af65b7ce270ba3e7 timeout expired in wait_until_healthy [6743921] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:15:38 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743921/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743921/ sentry: https://sentry.ceph.com/organizations/ceph/?query=050fac2e4d5e4cc3ae47de1fe950d76c timeout expired in wait_until_healthy [6743922] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:33:56 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743922/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743922/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e593924384b9466188fd214f07882851 timeout expired in wait_until_healthy [6743923] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:23:17 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743923/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743923/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4fccfb521484583ad0b2fde2ca519b3 timeout expired in wait_until_healthy [6743925] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:21:34 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743925/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743925/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fc98f84b84b2453db6787dde701b4841 timeout expired in wait_until_healthy [6743926] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} ----------------------------------------------------------------- time: 00:22:05 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743926/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743926/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e2b21dee58204f1dab53da60c34aaf4e timeout expired in wait_until_healthy [6743930] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:22:37 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743930/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743930/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4d41c5d1db148cb9c3775688d63bf8a timeout expired in wait_until_healthy [6743931] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:21:27 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743931/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743931/ sentry: https://sentry.ceph.com/organizations/ceph/?query=acce2eee21d7487db13be0eda0b85c58 timeout expired in wait_until_healthy [6743932] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:24:41 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743932/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743932/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4d62785275454367be4234ab1c11b48a timeout expired in wait_until_healthy [6743933] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:16:49 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743933/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743933/ sentry: https://sentry.ceph.com/organizations/ceph/?query=98b8d50c55c34787aeac5b9322d49dce timeout expired in wait_until_healthy [6743934] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:35:48 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743934/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743934/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3946c77ec14a43c596555646ba061159 timeout expired in wait_until_healthy [6743936] orch:cephadm/workunits/{agent/off mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:13:16 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743936/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743936/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c3f511de130e40c79e80e25d8be62310 Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [6743937] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:15:56 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743937/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743937/ sentry: https://sentry.ceph.com/organizations/ceph/?query=05ec63d9b8e343dc8d3a09a53ae2f307 timeout expired in wait_until_healthy [6743938] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:35:38 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743938/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743938/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0fea7feb37ed4dc4890c2dbc01b78670 timeout expired in wait_until_healthy [6743939] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final} ----------------------------------------------------------------- time: 00:15:29 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743939/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743939/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf6415d3460c4b1bb61703fcb2059439 timeout expired in wait_until_healthy [6743942] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:16:19 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743942/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743942/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f7cfc9aea26c4b9d8ed8fcdbafceb129 timeout expired in wait_until_healthy [6743944] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:20:50 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743944/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743944/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3dbc6d802ee74d05b14084da83285c8a timeout expired in wait_until_healthy [6743945] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:21:12 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743945/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743945/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5d6f88356536492d853085ee1f257e4d timeout expired in wait_until_healthy [6743946] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:23:03 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743946/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743946/ sentry: https://sentry.ceph.com/organizations/ceph/?query=57c175f226944d24a533bf132410d8c8 timeout expired in wait_until_healthy [6743947] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:34:14 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743947/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743947/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a0e09dcd22c648beb8577dcdc47cf073 timeout expired in wait_until_healthy [6743949] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:22:35 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743949/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743949/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1e220225cada4edda7ea5ac1ff5d4115 timeout expired in wait_until_healthy [6743950] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:23:34 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743950/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743950/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30e1b476d5f9479a9ff2aea76fee0b1e timeout expired in wait_until_healthy [6743951] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:15:56 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743951/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743951/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d503d30903df443488c6ef017c7e145a timeout expired in wait_until_healthy [6743952] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:16:30 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743952/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743952/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5c6a0fa134fa4806a6eeff107bc80e9a timeout expired in wait_until_healthy [6743956] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:15:36 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743956/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743956/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d459aa45cb4a476e96e42e3ec8752258 timeout expired in wait_until_healthy [6743957] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:20:44 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743957/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743957/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9929653c414c47ed88f2bb9309154c77 timeout expired in wait_until_healthy [6743960] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:20:36 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743960/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743960/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da2ce4a43b93429f9973d26c08b7098b timeout expired in wait_until_healthy [6743962] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:15:29 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743962/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743962/ sentry: https://sentry.ceph.com/organizations/ceph/?query=abb34782f7d04a5ba30a52f4ecac525b timeout expired in wait_until_healthy [6743963] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:16:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743963/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743963/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd33eab400ce44d4ba7a6cbb762a08e1 timeout expired in wait_until_healthy [6743965] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:21:42 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743965/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d38abb0665824147b564d5db5959ee61 timeout expired in wait_until_healthy [6743967] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:22:02 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743967/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743967/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b4d428f30e24098b130bda4e2051cb1 timeout expired in wait_until_healthy [6743969] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:34:47 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743969/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743969/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3379ef8c863b43509e3e97d49b81448d timeout expired in wait_until_healthy [6743972] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:21:20 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743972/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743972/ sentry: https://sentry.ceph.com/organizations/ceph/?query=78b8801a23e8403c8d77b44ef96358c8 timeout expired in wait_until_healthy [6743975] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} ----------------------------------------------------------------- time: 00:16:29 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743975/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743975/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5362b4ce917740cd9a0f5261c9163bf3 timeout expired in wait_until_healthy [6743976] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:34:01 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743976/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743976/ sentry: https://sentry.ceph.com/organizations/ceph/?query=68d880b757ac4730bc9bedb5aba92aaa timeout expired in wait_until_healthy [6743977] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:15:50 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743977/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743977/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad274a38c62440a5aabbe067b7ca9764 timeout expired in wait_until_healthy [6743978] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:21:13 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743978/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743978/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f6cc52b82194ca3842734c8db9cd6d5 timeout expired in wait_until_healthy [6743980] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:15:46 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743980/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743980/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0ba80c802d2549f79b14d23f4e84e7de timeout expired in wait_until_healthy [6743981] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:33:51 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743981/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743981/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b07b8eb6ec654ee58340e859c5f83bef timeout expired in wait_until_healthy [6743982] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:22:47 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743982/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743982/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28048e5c4e4f40fe8bf9c3a4e1cf4c3b timeout expired in wait_until_healthy [6743984] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:21:54 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743984/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743984/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50b9e0b04c02419e891ad640cb8cfa34 timeout expired in wait_until_healthy [6743987] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:14:35 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743987/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743987/ sentry: https://sentry.ceph.com/organizations/ceph/?query=99a413def1b746cd985f12a8e2a3acf8 Command failed on smithi040 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:93d090eac4a00adb26cea7ff2afbad83805cf2ef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3d5582a0-a68d-11ec-8c35-001a4aab830c -- ceph orch apply node-exporter '2;smithi025=a;smithi040=b'" [6743988] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} ----------------------------------------------------------------- time: 00:21:30 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743988/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743988/ sentry: https://sentry.ceph.com/organizations/ceph/?query=86c6f675d6424c6cad588ffb18f1f598 timeout expired in wait_until_healthy [6743989] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:21:38 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743989/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743989/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bb6c541b602742d69a62eca318059289 timeout expired in wait_until_healthy [6743990] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:24:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743990/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743990/ sentry: https://sentry.ceph.com/organizations/ceph/?query=85d41c84b94e43f2a3c4e7c0fef5ece0 timeout expired in wait_until_healthy Dead ================================================================= [6743911] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743911/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743911/ hit max job timeout [6743915] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743915/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743915/ hit max job timeout [6743927] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743927/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743927/ hit max job timeout [6743928] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743928/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743928/ hit max job timeout [6743940] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743940/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743940/ hit max job timeout [6743941] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743941/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743941/ hit max job timeout [6743943] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743943/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743943/ hit max job timeout [6743953] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743953/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743953/ hit max job timeout [6743959] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743959/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743959/ hit max job timeout [6743971] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743971/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743971/ hit max job timeout [6743973] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743973/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743973/ hit max job timeout [6743974] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743974/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743974/ hit max job timeout [6743985] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743985/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743985/ hit max job timeout [6743986] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743986/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743986/ hit max job timeout Pass ================================================================= [6743916] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli} time: 00:15:10 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743916/ [6743917] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:36:02 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743917/ [6743924] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:04:49 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743924/ [6743929] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:19 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743929/ [6743935] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:33:39 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743935/ [6743948] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_orch_cli} time: 00:13:54 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743948/ [6743954] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli} time: 00:26:47 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743954/ [6743955] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:13:45 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743955/ [6743958] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:30:30 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743958/ [6743961] orch:cephadm/workunits/{agent/on mon_election/classic task/test_adoption} time: 00:07:41 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743961/ [6743964] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:21:43 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743964/ [6743966] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:13:13 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743966/ [6743968] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:34:25 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743968/ [6743970] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm} time: 00:15:32 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743970/ [6743979] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:19:14 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743979/ [6743983] orch:cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos} time: 00:04:49 info: http://pulpito.ceph.com/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743983/