2022-06-08T14:40:33.805 INFO:root:teuthology version: 1.1.1.dev364+g40fff4b2
2022-06-08T14:40:33.806 INFO:teuthology.results:subset: '1/999'
2022-06-08T14:40:33.807 INFO:teuthology.results:seed: '8812'
2022-06-08T16:41:29.770 INFO:root:teuthology version: 1.1.1.dev364+g40fff4b2
2022-06-08T16:41:29.798 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-06-08T18:50:41.039 INFO:teuthology.results:Tests finished! gathering results...
2022-06-08T18:50:46.997 INFO:scrape:Found 90 jobs
2022-06-08T18:50:47.701 INFO:scrape:Found 3 distinct failure reasons
2022-06-08T18:50:47.702 INFO:scrape:Failure: Command failed on smithi080 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf0882ac-e73d-11ec-8410-001a4aab830c -- bash -c 'ceph fs set cephfs allow_standby_replay false'"
2022-06-08T18:50:47.702 INFO:scrape:3 jobs: ['6868299', '6868315', '6868255']
2022-06-08T18:50:47.703 INFO:scrape:suites intersection: ['3-upgrade-with-workload']
2022-06-08T18:50:47.703 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-nfs', '3-inline/no', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-06-08T18:50:47.704 INFO:scrape:
2022-06-08T18:50:47.704 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7db9c7225dd98539f619cceed2f8f049d3f70d7a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2022-06-08T18:50:47.705 INFO:scrape:1 jobs: ['6868253']
2022-06-08T18:50:47.705 INFO:scrape:suites: ['orch:cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}']
2022-06-08T18:50:47.706 INFO:scrape:
2022-06-08T18:50:47.706 INFO:scrape:Failure: Cannot connect to remote host smithi123
2022-06-08T18:50:47.706 INFO:scrape:1 jobs: ['6868306']
2022-06-08T18:50:47.707 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/repave-all}', 'orch:cephadm/osds/{0-distro/ubuntu_18.04']
2022-06-08T18:50:47.707 INFO:scrape:
2022-06-08T18:50:47.711 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/
failed:  5
dead:    0
running: 0
waiting: 0
queued:  0
passed:  85


Fail
=================================================================
[6868253]  orch:cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e}
-----------------------------------------------------------------
time:   00:33:41
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868253/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868253/
sentry: https://sentry.ceph.com/organizations/ceph/?query=30d742ee15094bdda0f4885c29eefc0e

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi083
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=7db9c7225dd98539f619cceed2f8f049d3f70d7a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[6868255]  orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:34:42
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868255/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868255/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2aa960613a0547f1bbd48023a3d4abb2

    Command failed on smithi149 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    53cdbe76-e73d-11ec-8410-001a4aab830c -e
    sha1=7db9c7225dd98539f619cceed2f8f049d3f70d7a -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[6868299]  orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:27:10
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868299/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868299/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8fd7ad8b178d42c4ac760c84d014fa2e

    Command failed on smithi080 with status 1: "sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    bf0882ac-e73d-11ec-8410-001a4aab830c -- bash -c 'ceph fs set cephfs
    allow_standby_replay false'"

[6868306]  orch:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868306/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868306/
sentry: https://sentry.ceph.com/organizations/ceph/?query=728fec3488bd4b3e95a2caae36ec5eba

    Cannot connect to remote host smithi123

[6868315]  orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:23:44
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868315/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868315/
sentry: https://sentry.ceph.com/organizations/ceph/?query=850e386bfea7453481fbc2ef7fb1489d

    Command failed on smithi002 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    fa4db0c4-e749-11ec-8410-001a4aab830c -e
    sha1=7db9c7225dd98539f619cceed2f8f049d3f70d7a -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''



Pass
=================================================================
[6868247] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:25:47
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868247/

[6868248] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls}
time:   00:50:26
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868248/

[6868249] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic}
time:   00:46:26
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868249/

[6868250] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:26:00
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868250/

[6868251] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
time:   00:44:56
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868251/

[6868252] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests}
time:   00:53:07
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868252/

[6868254] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:45:17
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868254/

[6868256] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli}
time:   00:31:43
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868256/

[6868257] orch:cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi}
time:   00:52:26
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868257/

[6868258] orch:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start}
time:   00:27:53
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868258/

[6868259] orch:cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/basic 3-final}
time:   00:24:52
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868259/

[6868260] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption}
time:   00:20:55
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868260/

[6868261] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   01:09:13
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868261/

[6868262] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:27:40
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868262/

[6868263] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api}
time:   00:50:21
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868263/

[6868264] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start}
time:   00:30:06
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868264/

[6868265] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm}
time:   00:26:16
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868265/

[6868266] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:25:52
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868266/

[6868267] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:43:32
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868267/

[6868268] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:29:13
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868268/

[6868269] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps}
time:   01:08:59
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868269/

[6868270] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
time:   00:50:51
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868270/

[6868271] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
time:   00:46:12
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868271/

[6868272] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:32:42
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868272/

[6868273] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start}
time:   00:28:16
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868273/

[6868274] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos}
time:   00:16:25
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868274/

[6868275] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench}
time:   01:46:34
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868275/

[6868276] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity}
time:   01:06:12
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868276/

[6868277] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:44:44
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868277/

[6868278] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:34:33
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868278/

[6868279] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls}
time:   00:40:54
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868279/

[6868280] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:27:14
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868280/

[6868281] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start}
time:   00:36:27
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868281/

[6868282] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs}
time:   00:35:32
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868282/

[6868283] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects}
time:   01:12:35
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868283/

[6868284] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:25:30
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868284/

[6868285] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
time:   00:48:39
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868285/

[6868286] orch:cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:54:11
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868286/

[6868287] orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:41:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868287/

[6868288] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:44:11
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868288/

[6868289] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
time:   00:50:54
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868289/

[6868290] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:28:52
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868290/

[6868291] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start}
time:   00:36:47
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868291/

[6868292] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli}
time:   00:23:42
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868292/

[6868293] orch:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:34:13
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868293/

[6868294] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps}
time:   01:00:42
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868294/

[6868295] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:31:27
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868295/

[6868296] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic}
time:   00:57:51
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868296/

[6868297] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
time:   01:08:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868297/

[6868298] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:42:37
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868298/

[6868300] orch:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start}
time:   00:28:10
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868300/

[6868301] orch:cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final}
time:   00:22:56
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868301/

[6868302] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption}
time:   00:20:34
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868302/

[6868303] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench}
time:   01:46:24
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868303/

[6868304] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:31:55
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868304/

[6868305] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
time:   00:36:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868305/

[6868307] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:15:16
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868307/

[6868308] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start}
time:   00:19:56
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868308/

[6868309] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm}
time:   00:17:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868309/

[6868310] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   00:59:18
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868310/

[6868311] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:33:02
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868311/

[6868312] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:14:51
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868312/

[6868313] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:32:56
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868313/

[6868314] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:36:29
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868314/

[6868316] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   00:26:05
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868316/

[6868317] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start}
time:   00:18:31
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868317/

[6868318] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos}
time:   00:08:41
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868318/

[6868319] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:18:57
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868319/

[6868320] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps}
time:   00:46:17
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868320/

[6868321] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:29:43
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868321/

[6868322] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:11
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868322/

[6868323] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:15:53
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868323/

[6868324] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench}
time:   02:09:32
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868324/

[6868325] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:19:11
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868325/

[6868326] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start}
time:   00:22:11
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868326/

[6868327] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs}
time:   00:27:10
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868327/

[6868328] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:37:41
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868328/

[6868329] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:35:50
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868329/

[6868330] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls}
time:   00:36:32
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868330/

[6868331] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:18:53
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868331/

[6868332] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:58
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868332/

[6868333] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start}
time:   00:20:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868333/

[6868334] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli}
time:   00:15:08
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868334/

[6868335] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:51:30
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868335/

[6868336] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:15:13
info:   http://pulpito.ceph.com/adking-2022-06-08_14:40:24-orch:cephadm-wip-adk2-testing-2022-06-07-2105-pacific-distro-default-smithi/6868336/