2022-04-29T12:07:47.889 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-29T12:07:47.891 INFO:teuthology.results:subset: '1/999' 2022-04-29T12:07:47.892 INFO:teuthology.results:seed: '3009' 2022-04-29T13:20:03.763 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-29T13:20:03.791 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-29T20:23:37.589 INFO:teuthology.results:Tests finished! gathering results... 2022-04-29T20:23:44.503 INFO:scrape:Found 90 jobs 2022-04-29T20:23:45.390 INFO:scrape:Found 3 distinct failure reasons 2022-04-29T20:23:45.391 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi185.front.sepia.ceph.com: ['type=AVC msg=audit(1651236114.879:17987): avc: denied { ioctl } for pid=106827 comm="iptables" path="/var/lib/containers/storage/overlay/9b7667cdae2519ad076e80d3c56871aa2d6c0b642b358d2c2fe22bae5058cfe1/merged" dev="overlay" ino=3412427 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1651236114.987:17990): avc: denied { ioctl } for pid=106844 comm="iptables" path="/var/lib/containers/storage/overlay/9b7667cdae2519ad076e80d3c56871aa2d6c0b642b358d2c2fe22bae5058cfe1/merged" dev="overlay" ino=3412427 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-04-29T20:23:45.391 INFO:scrape:2 jobs: ['6813147', '6813201'] 2022-04-29T20:23:45.391 INFO:scrape:suites intersection: ['rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-29T20:23:45.391 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-29T20:23:45.391 INFO:scrape: 2022-04-29T20:23:45.391 INFO:scrape:Dead: b'2022-04-29T19:00:15.942 DEBUG:teuthology.exit:Finished running handlers' 2022-04-29T20:23:45.392 INFO:scrape:2 jobs: ['6813149', '6813169'] 2022-04-29T20:23:45.392 INFO:scrape:suites intersection: ['3-upgrade-with-workload'] 2022-04-29T20:23:45.392 INFO:scrape:suites union: ['1-bootstrap/16.2.5', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-nfs', '3-inline/yes', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'roles', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-29T20:23:45.392 INFO:scrape: 2022-04-29T20:23:45.392 INFO:scrape:Failure: Command failed on smithi170 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fba84d44-c7be-11ec-8c39-001a4aab830c -e sha1=f5a42d88b7f0c145d50ecadc0b0ab8e041152e80 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2022-04-29T20:23:45.392 INFO:scrape:2 jobs: ['6813207', '6813136'] 2022-04-29T20:23:45.392 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2022-04-29T20:23:45.393 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2022-04-29T20:23:45.393 INFO:scrape: 2022-04-29T20:23:45.396 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/ failed: 4 dead: 2 running: 0 waiting: 0 queued: 0 passed: 84 Fail ================================================================= [6813136] rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:22:00 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813136/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813136/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ff5cbc86b024d15b34a4f849aa32901 Command failed on smithi120 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c5f7999a-c7b7-11ec-8c39-001a4aab830c -e sha1=f5a42d88b7f0c145d50ecadc0b0ab8e041152e80 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6813147] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:14:37 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813147/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813147/ SELinux denials found on ubuntu@smithi185.front.sepia.ceph.com: ['type=AVC msg=audit(1651236114.879:17987): avc: denied { ioctl } for pid=106827 comm="iptables" path="/var/lib/containers/storage/overlay/9b7667cdae2519ad0 76e80d3c56871aa2d6c0b642b358d2c2fe22bae5058cfe1/merged" dev="overlay" ino=3412427 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1651236114.987:17990): avc: denied { ioctl } for pid=106844 comm="iptables" path="/var/lib/containers/storage/o verlay/9b7667cdae2519ad076e80d3c56871aa2d6c0b642b358d2c2fe22bae5058cfe1/mer ged" dev="overlay" ino=3412427 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6813201] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:15:06 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813201/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813201/ SELinux denials found on ubuntu@smithi039.front.sepia.ceph.com: ['type=AVC msg=audit(1651237673.645:17996): avc: denied { ioctl } for pid=106774 comm="iptables" path="/var/lib/containers/storage/overlay/bbe6427afa935d364 3c44a09c4f5d70567da15ad45e41222f1ad37660336507d/merged" dev="overlay" ino=3412409 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6813207] rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:21:24 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813207/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813207/ sentry: https://sentry.ceph.com/organizations/ceph/?query=411c88e2daac437bbcad70b690d9f6b2 Command failed on smithi170 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fba84d44-c7be-11ec-8c39-001a4aab830c -e sha1=f5a42d88b7f0c145d50ecadc0b0ab8e041152e80 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' Dead ================================================================= [6813149] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813149/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813149/ hit max job timeout [6813169] rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813169/ log: http://qa-proxy.ceph.com/teuthology/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813169/ hit max job timeout Pass ================================================================= [6813129] rados:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:12:11 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813129/ [6813130] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:28:57 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813130/ [6813131] rados:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:21 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813131/ [6813132] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:32:13 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813132/ [6813133] rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:47:36 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813133/ [6813134] rados:cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:22:20 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813134/ [6813135] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:27 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813135/ [6813137] rados:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr orchestrator_cli} time: 00:11:22 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813137/ [6813138] rados:cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:34:06 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813138/ [6813139] rados:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:14:39 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813139/ [6813140] rados:cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:10:54 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813140/ [6813141] rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:30:37 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813141/ [6813142] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:10:07 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813142/ [6813143] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:45:06 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813143/ [6813144] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:36 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813144/ [6813145] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:27:45 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813145/ [6813146] rados:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:18:43 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813146/ [6813148] rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:34 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813148/ [6813150] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:17:27 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813150/ [6813151] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:39:37 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813151/ [6813152] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:36:08 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813152/ [6813153] rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:32:57 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813153/ [6813154] rados:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:18:09 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813154/ [6813155] rados:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:17:34 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813155/ [6813156] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:06:49 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813156/ [6813157] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:36:04 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813157/ [6813158] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:22 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813158/ [6813159] rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:30:35 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813159/ [6813160] rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:21:33 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813160/ [6813161] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:21:45 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813161/ [6813162] rados:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:56 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813162/ [6813163] rados:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:18:18 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813163/ [6813164] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:25:02 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813164/ [6813165] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:49:40 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813165/ [6813166] rados:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:46 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813166/ [6813167] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:34:49 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813167/ [6813168] rados:cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:36:54 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813168/ [6813170] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:22 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813170/ [6813171] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:28:04 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813171/ [6813172] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:17:23 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813172/ [6813173] rados:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:19:52 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813173/ [6813174] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:13:09 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813174/ [6813175] rados:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:19 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813175/ [6813176] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:40:15 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813176/ [6813177] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:18:50 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813177/ [6813178] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:46:09 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813178/ [6813180] rados:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:28:03 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813180/ [6813182] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:25 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813182/ [6813184] rados:cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:14:47 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813184/ [6813186] rados:cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:12:13 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813186/ [6813188] rados:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:24:51 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813188/ [6813189] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:10:02 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813189/ [6813191] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench} time: 01:19:09 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813191/ [6813193] rados:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:17:19 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813193/ [6813195] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:29:19 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813195/ [6813197] rados:cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:31 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813197/ [6813198] rados:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:16:17 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813198/ [6813200] rados:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:18:51 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813200/ [6813202] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:45:26 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813202/ [6813203] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:29 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813203/ [6813204] rados:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:10 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813204/ [6813205] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:30:06 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813205/ [6813206] rados:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:34:06 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813206/ [6813208] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:21:27 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813208/ [6813209] rados:cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:17:48 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813209/ [6813210] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:06:55 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813210/ [6813211] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:17:22 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813211/ [6813212] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:39:35 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813212/ [6813213] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:55 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813213/ [6813214] rados:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:31:18 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813214/ [6813215] rados:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:08 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813215/ [6813216] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:56:02 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813216/ [6813217] rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:40 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813217/ [6813218] rados:cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:18:17 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813218/ [6813219] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:25:48 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813219/ [6813220] rados:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:37:00 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813220/ [6813221] rados:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:35:44 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813221/ [6813222] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:29:16 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813222/ [6813223] rados:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:17:13 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813223/ [6813224] rados:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:27 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813224/ [6813225] rados:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:20:40 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813225/ [6813226] rados:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:13:43 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813226/ [6813227] rados:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:44:47 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813227/ [6813228] rados:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:12:49 info: http://pulpito.ceph.com/adking-2022-04-29_12:06:52-rados:cephadm-wip-adk3-testing-2022-04-28-2054-pacific-distro-basic-smithi/6813228/