2022-05-22T00:08:03.421 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01
2022-05-22T00:08:03.423 INFO:teuthology.results:subset: '1/999'
2022-05-22T00:08:03.424 INFO:teuthology.results:seed: '3093'
2022-05-22T00:57:54.975 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01
2022-05-22T00:57:54.979 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-05-22T02:59:05.255 INFO:teuthology.results:Tests finished! gathering results...
2022-05-22T02:59:12.084 INFO:scrape:Found 90 jobs
2022-05-22T02:59:12.841 INFO:scrape:Found 3 distinct failure reasons
2022-05-22T02:59:12.843 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi083.front.sepia.ceph.com: ['type=AVC msg=audit(1653179663.839:18141): avc:  denied  { ioctl } for  pid=108295 comm="iptables" path="/var/lib/containers/storage/overlay/08c777f88da2584074f391d21508ee568622e146e3ed3470573f5b2d527c0b47/merged" dev="overlay" ino=3409836 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1653179663.707:18139): avc:  denied  { ioctl } for  pid=108280 comm="iptables" path="/var/lib/containers/storage/overlay/08c777f88da2584074f391d21508ee568622e146e3ed3470573f5b2d527c0b47/merged" dev="overlay" ino=3409836 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']
2022-05-22T02:59:12.844 INFO:scrape:1 jobs: ['6843456']
2022-05-22T02:59:12.845 INFO:scrape:suites: ['mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2022-05-22T02:59:12.846 INFO:scrape:
2022-05-22T02:59:12.846 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi082.front.sepia.ceph.com: ['type=AVC msg=audit(1653181130.504:18153): avc:  denied  { ioctl } for  pid=108328 comm="iptables" path="/var/lib/containers/storage/overlay/a0e1ab34732ba0cf0adb4b74999c40af3f8126ed79b47b08792886fc002e240d/merged" dev="overlay" ino=3409837 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1653181130.278:18150): avc:  denied  { ioctl } for  pid=108294 comm="iptables" path="/var/lib/containers/storage/overlay/a0e1ab34732ba0cf0adb4b74999c40af3f8126ed79b47b08792886fc002e240d/merged" dev="overlay" ino=3409837 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']
2022-05-22T02:59:12.848 INFO:scrape:1 jobs: ['6843500']
2022-05-22T02:59:12.849 INFO:scrape:suites: ['mon_election/classic', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2022-05-22T02:59:12.850 INFO:scrape:
2022-05-22T02:59:12.851 INFO:scrape:Failure: Command failed on smithi053 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fb2ec4a-d965-11ec-be40-001a4aab830c -e sha1=d3501fe9c13f2c2e896360776208e3d773c2f592 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2022-05-22T02:59:12.852 INFO:scrape:2 jobs: ['6843446', '6843506']
2022-05-22T02:59:12.853 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools']
2022-05-22T02:59:12.854 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools']
2022-05-22T02:59:12.855 INFO:scrape:
2022-05-22T02:59:12.860 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/
failed:  4
dead:    0
running: 0
waiting: 0
queued:  0
passed:  86


Fail
=================================================================
[6843446]  orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:24:58
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843446/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843446/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a300d8a6ecb4f4daec2c9d2f3553698

    Command failed on smithi053 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    7fb2ec4a-d965-11ec-be40-001a4aab830c -e
    sha1=d3501fe9c13f2c2e896360776208e3d773c2f592 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[6843456]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:19:33
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843456/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843456/

    SELinux denials found on ubuntu@smithi083.front.sepia.ceph.com: ['type=AVC
    msg=audit(1653179663.839:18141): avc:  denied  { ioctl } for  pid=108295
    comm="iptables" path="/var/lib/containers/storage/overlay/08c777f88da258407
    4f391d21508ee568622e146e3ed3470573f5b2d527c0b47/merged" dev="overlay"
    ino=3409836 scontext=system_u:system_r:iptables_t:s0
    tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir
    permissive=1', 'type=AVC msg=audit(1653179663.707:18139): avc:  denied  {
    ioctl } for  pid=108280 comm="iptables" path="/var/lib/containers/storage/o
    verlay/08c777f88da2584074f391d21508ee568622e146e3ed3470573f5b2d527c0b47/mer
    ged" dev="overlay" ino=3409836 scontext=system_u:system_r:iptables_t:s0
    tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir
    permissive=1']

[6843500]  orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:15:57
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843500/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843500/

    SELinux denials found on ubuntu@smithi082.front.sepia.ceph.com: ['type=AVC
    msg=audit(1653181130.504:18153): avc:  denied  { ioctl } for  pid=108328
    comm="iptables" path="/var/lib/containers/storage/overlay/a0e1ab34732ba0cf0
    adb4b74999c40af3f8126ed79b47b08792886fc002e240d/merged" dev="overlay"
    ino=3409837 scontext=system_u:system_r:iptables_t:s0
    tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir
    permissive=1', 'type=AVC msg=audit(1653181130.278:18150): avc:  denied  {
    ioctl } for  pid=108294 comm="iptables" path="/var/lib/containers/storage/o
    verlay/a0e1ab34732ba0cf0adb4b74999c40af3f8126ed79b47b08792886fc002e240d/mer
    ged" dev="overlay" ino=3409837 scontext=system_u:system_r:iptables_t:s0
    tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir
    permissive=1']

[6843506]  orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:22:08
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843506/
log:    http://qa-proxy.ceph.com/teuthology/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843506/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b5ff5add1bdd410bbc0722ce2820ec3c

    Command failed on smithi003 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    91190a9c-d969-11ec-be40-001a4aab830c -e
    sha1=d3501fe9c13f2c2e896360776208e3d773c2f592 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''



Pass
=================================================================
[6843438] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:21:05
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843438/

[6843439] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls}
time:   00:42:55
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843439/

[6843440] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic}
time:   00:39:13
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843440/

[6843441] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:20:25
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843441/

[6843442] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
time:   00:40:15
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843442/

[6843443] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests}
time:   00:55:36
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843443/

[6843444] orch:cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e}
time:   00:28:00
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843444/

[6843445] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:36:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843445/

[6843447] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr orchestrator_cli}
time:   00:21:19
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843447/

[6843448] orch:cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi}
time:   00:31:47
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843448/

[6843449] orch:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start}
time:   00:23:13
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843449/

[6843450] orch:cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final}
time:   00:20:56
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843450/

[6843451] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption}
time:   00:13:54
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843451/

[6843452] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   00:57:20
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843452/

[6843453] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:24:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843453/

[6843454] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api}
time:   00:40:32
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843454/

[6843455] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start}
time:   00:28:43
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843455/

[6843457] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:23:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843457/

[6843458] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:35:57
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843458/

[6843459] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:21:17
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843459/

[6843460] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps}
time:   01:03:36
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843460/

[6843461] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
time:   00:45:07
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843461/

[6843462] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
time:   00:42:39
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843462/

[6843463] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:24:02
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843463/

[6843464] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start}
time:   00:26:05
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843464/

[6843465] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos}
time:   00:11:13
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843465/

[6843466] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench}
time:   01:47:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843466/

[6843467] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:59:43
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843467/

[6843468] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:38:51
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843468/

[6843469] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:25:22
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843469/

[6843470] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls}
time:   00:35:58
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843470/

[6843471] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:22:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843471/

[6843472] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start}
time:   00:23:04
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843472/

[6843473] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs}
time:   00:29:37
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843473/

[6843474] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects}
time:   00:58:37
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843474/

[6843475] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:19:09
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843475/

[6843476] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
time:   00:43:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843476/

[6843477] orch:cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:45:06
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843477/

[6843478] orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:34:43
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843478/

[6843479] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:35:02
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843479/

[6843480] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
time:   00:41:51
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843480/

[6843481] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:23:49
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843481/

[6843482] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start}
time:   00:30:01
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843482/

[6843483] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli}
time:   00:18:46
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843483/

[6843484] orch:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:25:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843484/

[6843485] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps}
time:   00:50:51
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843485/

[6843486] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:26:16
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843486/

[6843487] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic}
time:   00:52:08
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843487/

[6843488] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
time:   01:00:12
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843488/

[6843489] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:37:53
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843489/

[6843490] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:37:58
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843490/

[6843491] orch:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start}
time:   00:23:46
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843491/

[6843492] orch:cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/rgw 3-final}
time:   00:19:52
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843492/

[6843493] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption}
time:   00:15:00
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843493/

[6843494] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench}
time:   02:06:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843494/

[6843495] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:16:56
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843495/

[6843496] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
time:   00:32:58
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843496/

[6843497] orch:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:14:32
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843497/

[6843498] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:16:21
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843498/

[6843499] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start}
time:   00:21:24
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843499/

[6843501] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   00:50:10
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843501/

[6843502] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:14
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843502/

[6843503] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:13:42
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843503/

[6843504] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:32:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843504/

[6843505] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:36:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843505/

[6843507] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   00:25:38
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843507/

[6843508] orch:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start}
time:   00:20:06
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843508/

[6843509] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos}
time:   00:07:24
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843509/

[6843510] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:18:00
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843510/

[6843511] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps}
time:   00:42:49
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843511/

[6843512] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:32:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843512/

[6843513] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:29:11
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843513/

[6843514] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:01
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843514/

[6843515] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench}
time:   01:57:14
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843515/

[6843516] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:17:16
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843516/

[6843517] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start}
time:   00:20:55
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843517/

[6843518] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs}
time:   00:26:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843518/

[6843519] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:39:17
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843519/

[6843520] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:38:00
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843520/

[6843521] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls}
time:   00:34:11
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843521/

[6843522] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:17:08
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843522/

[6843523] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:21
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843523/

[6843524] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start}
time:   00:22:25
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843524/

[6843525] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli}
time:   00:14:27
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843525/

[6843526] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:49:01
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843526/

[6843527] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:13:37
info:   http://pulpito.ceph.com/adking-2022-05-22_00:07:49-orch:cephadm-wip-adk2-testing-2022-05-21-1835-pacific-distro-basic-smithi/6843527/