2022-09-03T07:50:45.310 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-03T07:50:45.312 INFO:teuthology.results:subset: '1/6' 2022-09-03T07:50:45.312 INFO:teuthology.results:seed: '7228' 2022-09-03T09:01:32.141 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-03T09:01:32.169 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-09-03T09:59:37.348 INFO:teuthology.results:Tests finished! gathering results... 2022-09-03T09:59:44.256 INFO:scrape:Found 90 jobs 2022-09-03T09:59:46.285 INFO:scrape:Found 4 distinct failure reasons 2022-09-03T09:59:46.287 INFO:scrape:Failure: Command failed on smithi012 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8434c9e-2b65-11ed-8431-001a4aab830c -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph | grep 16.2.0'" 2022-09-03T09:59:46.288 INFO:scrape:4 jobs: ['7005854', '7005809', '7005829', '7005874'] 2022-09-03T09:59:46.289 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', '5-upgrade-ls'] 2022-09-03T09:59:46.290 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', '5-upgrade-ls', 'agent/off', 'agent/on', 'mon_election/classic}', 'mon_election/connectivity}', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9'] 2022-09-03T09:59:46.290 INFO:scrape: 2022-09-03T09:59:46.291 INFO:scrape:Failure: Command failed on smithi116 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' 2022-09-03T09:59:46.292 INFO:scrape:1 jobs: ['7005805'] 2022-09-03T09:59:46.292 INFO:scrape:suites: ['base/install', 'cluster/{fixed-3', 'openstack}', 'orch:cephadm/rbd_iscsi/{0-single-container-host', 'workloads/cephadm_iscsi}'] 2022-09-03T09:59:46.293 INFO:scrape: 2022-09-03T09:59:46.293 INFO:scrape:Failure: Command failed on smithi154 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.154:/foouser /mnt/foo'" 2022-09-03T09:59:46.293 INFO:scrape:2 jobs: ['7005846', '7005842'] 2022-09-03T09:59:46.294 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '3-final}'] 2022-09-03T09:59:46.295 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '3-final}', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04'] 2022-09-03T09:59:46.295 INFO:scrape: 2022-09-03T09:59:46.296 INFO:scrape:Failure: Command failed (workunit test cephadm/test_repos.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8e7f49c256f8f4423de0179cd5ade14f6f211bd5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' 2022-09-03T09:59:46.297 INFO:scrape:2 jobs: ['7005811', '7005876'] 2022-09-03T09:59:46.298 INFO:scrape:suites intersection: ['task/test_cephadm_repos}'] 2022-09-03T09:59:46.299 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'orch:cephadm/workunits/{agent/off', 'orch:cephadm/workunits/{agent/on', 'task/test_cephadm_repos}'] 2022-09-03T09:59:46.299 INFO:scrape: 2022-09-03T09:59:46.303 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/ failed: 9 dead: 0 running: 0 waiting: 0 queued: 0 passed: 81 Fail ================================================================= [7005805] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} ----------------------------------------------------------------- time: 00:31:04 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005805/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005805/ sentry: https://sentry.ceph.com/organizations/ceph/?query=253691456a2e487c8babae51f456b8b5 Command failed on smithi116 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [7005809] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:35:36 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005809/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005809/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a58831b46f544f91896fe774bc50ab02 Command failed on smithi006 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0f4e3792-2b5f-11ed-8431-001a4aab830c -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph | grep 16.2.0'" [7005811] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:06:27 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005811/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005811/ sentry: https://sentry.ceph.com/organizations/ceph/?query=752e2c7f0ca5456992ef275f933332db Command failed (workunit test cephadm/test_repos.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8e7f49c256f8f4423de0179cd5ade14f6f211bd5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7005829] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} ----------------------------------------------------------------- time: 00:34:10 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005829/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005829/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ac14148ac0c048c6a078e5d28f0cb84c Command failed on smithi112 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9da91bc8-2b60-11ed-8431-001a4aab830c -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph | grep 16.2.0'" [7005842] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:19:30 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005842/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005842/ sentry: https://sentry.ceph.com/organizations/ceph/?query=602f579e1d4841c1a2faa186baf03041 Command failed on smithi006 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.6:/foobucket /mnt/foo'" [7005846] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:22:56 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005846/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005846/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec0cd3b9e4354ee39199a6196be9315b Command failed on smithi154 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.154:/foouser /mnt/foo'" [7005854] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:28:06 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005854/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005854/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d07d41e60fdc4767b1be36829389971e Command failed on smithi012 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8434c9e-2b65-11ed-8431-001a4aab830c -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph | grep 16.2.0'" [7005874] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:34:45 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005874/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9df5984a9e194d36b6c32d05415a4fdc Command failed on smithi071 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 726d3780-2b67-11ed-8431-001a4aab830c -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph | grep 16.2.0'" [7005876] orch:cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:05:57 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005876/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005876/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f6c3ff82d13487e96b5116011093ba4 Command failed (workunit test cephadm/test_repos.sh) on smithi026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8e7f49c256f8f4423de0179cd5ade14f6f211bd5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' Pass ================================================================= [7005801] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:33:12 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005801/ [7005802] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:10 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005802/ [7005803] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:39:35 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005803/ [7005804] orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} time: 00:15:35 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005804/ [7005806] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:17:09 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005806/ [7005807] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.5_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:15:31 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005807/ [7005808] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:34:43 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005808/ [7005810] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:38:55 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005810/ [7005812] orch:cephadm/smoke/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:51 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005812/ [7005813] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:16:15 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005813/ [7005814] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:26:07 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005814/ [7005815] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:14 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005815/ [7005816] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:20:31 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005816/ [7005817] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:39:28 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005817/ [7005818] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:40:14 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005818/ [7005819] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:06 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005819/ [7005820] orch:cephadm/workunits/{agent/on mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:17:03 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005820/ [7005821] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:16:38 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005821/ [7005822] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:26:13 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005822/ [7005823] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:27 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005823/ [7005824] orch:cephadm/smoke/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:12 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005824/ [7005825] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:25 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005825/ [7005826] orch:cephadm/thrash/{0-distro/rhel_8.5_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:40:04 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005826/ [7005827] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:44:09 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005827/ [7005828] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:14:45 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005828/ [7005830] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:25:47 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005830/ [7005831] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_nfs} time: 00:33:04 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005831/ [7005832] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:15:51 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005832/ [7005833] orch:cephadm/thrash/{0-distro/rhel_8.5_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:55:38 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005833/ [7005834] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:38:12 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005834/ [7005835] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:54 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005835/ [7005836] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:09 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005836/ [7005837] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:34:33 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005837/ [7005838] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:25:37 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005838/ [7005839] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:46 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005839/ [7005840] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:27:15 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005840/ [7005841] orch:cephadm/workunits/{agent/on mon_election/classic task/test_orch_cli} time: 00:19:03 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005841/ [7005843] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:43:58 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005843/ [7005844] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:38:32 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005844/ [7005845] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:32 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005845/ [7005847] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:31:37 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005847/ [7005848] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:41 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005848/ [7005849] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.5_container_tools_rhel8} 2-node-mgr agent/on orchestrator_cli} time: 00:17:03 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005849/ [7005850] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:18:41 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005850/ [7005851] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:18:26 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005851/ [7005852] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.5_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:16:57 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005852/ [7005853] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:38:31 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005853/ [7005855] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:37:42 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005855/ [7005856] orch:cephadm/workunits/{agent/on mon_election/classic task/test_adoption} time: 00:08:47 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005856/ [7005857] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:25 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005857/ [7005858] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:25:30 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005858/ [7005859] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:21 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005859/ [7005860] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:15:41 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005860/ [7005861] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:36:31 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005861/ [7005862] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:38:24 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005862/ [7005863] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm} time: 00:16:59 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005863/ [7005864] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:17:24 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005864/ [7005865] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:33 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005865/ [7005866] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:25:04 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005866/ [7005867] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:38:54 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005867/ [7005868] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:33 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005868/ [7005869] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:37 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005869/ [7005870] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:24:06 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005870/ [7005871] orch:cephadm/thrash/{0-distro/rhel_8.5_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:58:52 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005871/ [7005872] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:44:26 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005872/ [7005873] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:17:22 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005873/ [7005875] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:25:12 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005875/ [7005877] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:22 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005877/ [7005878] orch:cephadm/smoke/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:19:32 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005878/ [7005879] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:14:46 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005879/ [7005880] orch:cephadm/thrash/{0-distro/rhel_8.5_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:32:44 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005880/ [7005881] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:38:39 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005881/ [7005882] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:15:55 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005882/ [7005883] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:27:38 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005883/ [7005884] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:11 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005884/ [7005885] orch:cephadm/workunits/{agent/off mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:15:48 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005885/ [7005886] orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:56 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005886/ [7005887] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:50:06 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005887/ [7005888] orch:cephadm/with-work/{0-distro/rhel_8.5_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:37:06 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005888/ [7005889] orch:cephadm/osds/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:28 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005889/ [7005890] orch:cephadm/smoke/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:16:44 info: http://pulpito.ceph.com/adking-2022-09-01_18:55:50-orch:cephadm-main-distro-default-smithi/7005890/