2022-04-24T06:08:41.146 INFO:root:teuthology version: 1.1.1.dev313+g1b11219b 2022-04-24T06:08:41.147 INFO:teuthology.results:subset: '17775/99999' 2022-04-24T06:08:41.147 INFO:teuthology.results:seed: '7' 2022-04-24T11:12:10.417 INFO:root:teuthology version: 1.1.1.dev313+g1b11219b 2022-04-24T11:12:10.445 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-24T17:16:44.826 INFO:teuthology.results:Tests finished! gathering results... 2022-04-24T17:17:17.228 INFO:scrape:Found 413 jobs 2022-04-24T17:17:29.832 INFO:scrape:Found 11 distinct failure reasons 2022-04-24T17:17:29.833 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi016.front.sepia.ceph.com: ['type=AVC msg=audit(1650783593.578:7452): avc: denied { ioctl } for pid=49931 comm="iptables" path="/var/lib/containers/storage/overlay/aafdabf96286cbd0c8143e515d684b0fe214dec80ce8745a515764b1c2aee71b/merged" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1650783593.680:7454): avc: denied { ioctl } for pid=49949 comm="iptables" path="/var/lib/containers/storage/overlay/aafdabf96286cbd0c8143e515d684b0fe214dec80ce8745a515764b1c2aee71b/merged" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-04-24T17:17:29.833 INFO:scrape:1 jobs: ['6803237'] 2022-04-24T17:17:29.833 INFO:scrape:suites: ['mon_election/classic', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-24T17:17:29.834 INFO:scrape: 2022-04-24T17:17:29.834 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi165.front.sepia.ceph.com: ['type=AVC msg=audit(1650794012.596:200): avc: denied { node_bind } for pid=2007 comm="ping" saddr=172.21.15.165 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2022-04-24T17:17:29.834 INFO:scrape:35 jobs 2022-04-24T17:17:29.834 INFO:scrape:suites intersection: [] 2022-04-24T17:17:29.835 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/octopus', '1-ranks/1', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-nfs', '2-ops/rm-zap-wait}', '2-repo_digest/repo_digest', '2-services/rgw', '2-thrash', '3-final}', '3-inline/yes', '3-start-upgrade', '3-tasks/rados_api_tests', '3-tasks/snaps-few-objects', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', '4-wait', '5-upgrade-ls', 'centos_8.stream_container_tools', 'ceph', 'clusters/9-mons', 'clusters/{fixed-2', 'conf/{client', 'd-thrash/none', 'debug/mgr', 'fast/normal', 'fixed-2', 'mds', 'mgr_ttl_cache/enable', 'mode/packaged', 'mon', 'mon_election/classic', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'osd}', 'overrides/{pg-warn', 'rados', 'rados/basic/{ceph', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/osds/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/monthrash/{ceph', 'rados/objectstore/{backends/ceph_objectstore_tool', 'rados/rest/{mgr-restful', 'rados/singleton-nomsgr/{all/ceph-post-file', 'rados/singleton-nomsgr/{all/export-after-evict', 'rados/singleton-nomsgr/{all/health-warnings', 'rados/singleton/{all/divergent_priors', 'rados/singleton/{all/dump-stuck', 'rados/singleton/{all/ec-lost-unfound', 'rados/singleton/{all/mon-auth-caps', 'rados/singleton/{all/pg-removal-interruption', 'rados/singleton/{all/rebuild-mondb', 'rados/singleton/{all/recovery-preemption', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code/{ceph', 'rados/verify/{centos_latest', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'roles', 'root}', 'start', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'task/test_adoption}', 'task/test_cephadm_repos}', 'task/test_cephadm}', 'task/test_orch_cli}', 'tasks/failover}', 'tasks/insights}', 'tasks/mon_recovery', 'tasks/rados_python}', 'tasks/rados_workunit_loadgen_mix}', 'tasks/{0-from/v16.2.4', 'thrashers/force-sync-many', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'validater/lockdep}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-small-objects-balanced}', 'workloads/snaps-few-objects}'] 2022-04-24T17:17:29.835 INFO:scrape: 2022-04-24T17:17:29.835 INFO:scrape:Failure: SSH connection to smithi080 was lost: 'uname -r' 2022-04-24T17:17:29.835 INFO:scrape:9 jobs: ['6803241', '6803384', '6803434', '6803493', '6803549', '6803598', '6803471', '6803251', '6803519'] 2022-04-24T17:17:29.835 INFO:scrape:suites intersection: [] 2022-04-24T17:17:29.836 INFO:scrape:suites union: ['0-kubeadm', '1-install/luminous-v1only', '1-install/mimic', '1-install/mimic-v1only', '1-install/nautilus-v2only', '1-rook', '2-workload/none', '3-final', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'cluster/1-node', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'distro/ubuntu_18.04', 'fixed-2', 'k8s/1.21', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'net/calico', 'rados', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/with-work/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rook/master}', 'start', 'start}', 'tasks/rados_api_tests}', 'thrashers/careful', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] 2022-04-24T17:17:29.836 INFO:scrape: 2022-04-24T17:17:29.836 INFO:scrape:Failure: "2022-04-24T10:45:43.564172+0000 mon.a (mon.0) 10 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log 2022-04-24T17:17:29.836 INFO:scrape:1 jobs: ['6803555'] 2022-04-24T17:17:29.836 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/none', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'rados', 'rados/singleton/{all/backfill-toofull', 'supported-random-distro$/{ubuntu_latest}}'] 2022-04-24T17:17:29.836 INFO:scrape: 2022-04-24T17:17:29.837 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi003.front.sepia.ceph.com: ['type=AVC msg=audit(1650793308.176:7457): avc: denied { ioctl } for pid=50206 comm="iptables" path="/var/lib/containers/storage/overlay/459f4154bcb73059d217bdda508e23ebebc0dd2007f2e7413dee6e7cb6f87863/merged" dev="overlay" ino=3539055 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-04-24T17:17:29.837 INFO:scrape:1 jobs: ['6803444'] 2022-04-24T17:17:29.837 INFO:scrape:suites: ['mon_election/classic', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-24T17:17:29.837 INFO:scrape: 2022-04-24T17:17:29.837 INFO:scrape:Failure: Command failed on smithi002 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:b42bd433b8c85da129c1c277665f5ddab82dd5fb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88647a0a-c3aa-11ec-8c39-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-04-24T17:17:29.837 INFO:scrape:1 jobs: ['6803372'] 2022-04-24T17:17:29.837 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2022-04-24T17:17:29.838 INFO:scrape: 2022-04-24T17:17:29.838 INFO:scrape:Dead: b'2022-04-24T17:13:36.523 DEBUG:teuthology.exit:Finished running handlers' 2022-04-24T17:17:29.838 INFO:scrape:3 jobs: ['6803566', '6803379', '6803295'] 2022-04-24T17:17:29.838 INFO:scrape:suites intersection: [] 2022-04-24T17:17:29.838 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-flag}', 'ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'distro/ubuntu_20.04', 'fixed-2', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/verify/{centos_latest', 'start}', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2022-04-24T17:17:29.838 INFO:scrape: 2022-04-24T17:17:29.839 INFO:scrape:Failure: Command failed on smithi139 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9662f5e2-c3b8-11ec-8c39-001a4aab830c -e sha1=b42bd433b8c85da129c1c277665f5ddab82dd5fb -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2022-04-24T17:17:29.839 INFO:scrape:3 jobs: ['6803526', '6803254', '6803321'] 2022-04-24T17:17:29.839 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2022-04-24T17:17:29.839 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2022-04-24T17:17:29.839 INFO:scrape: 2022-04-24T17:17:29.839 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi101.front.sepia.ceph.com: ['type=AVC msg=audit(1650788751.830:202): avc: denied { node_bind } for pid=2089 comm="ping" saddr=172.21.15.101 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1650789462.507:7425): avc: denied { ioctl } for pid=50043 comm="iptables" path="/var/lib/containers/storage/overlay/8c6cccf5bcb34c4f4f7e2a26226bb59c7fa8f405043819a47e73a3ba921088cd/merged" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-04-24T17:17:29.839 INFO:scrape:1 jobs: ['6803347'] 2022-04-24T17:17:29.840 INFO:scrape:suites: ['mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-24T17:17:29.840 INFO:scrape: 2022-04-24T17:17:29.840 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi038.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2022-04-24T17:17:29.840 INFO:scrape:1 jobs: ['6803259'] 2022-04-24T17:17:29.840 INFO:scrape:suites: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rook/master}'] 2022-04-24T17:17:29.840 INFO:scrape: 2022-04-24T17:17:29.841 INFO:scrape:Failure: {'smithi039.front.sepia.ceph.com': {'_ansible_no_log': False, 'attempts': 24, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'only_upgrade': False, 'package': None, 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': True, 'upgrade': None}}, 'msg': 'Failed to update apt cache: '}} 2022-04-24T17:17:29.841 INFO:scrape:1 jobs: ['6803548'] 2022-04-24T17:17:29.841 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/connectivity', 'msgr-failures/mon-delay', 'msgr/async', 'objectstore/filestore-xfs', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/sync', 'workloads/snaps-few-objects}'] 2022-04-24T17:17:29.841 INFO:scrape: 2022-04-24T17:17:29.856 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/ failed: 44 dead: 13 running: 0 waiting: 0 queued: 0 passed: 356 Fail ================================================================= [6803237] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803237/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803237/ SELinux denials found on ubuntu@smithi016.front.sepia.ceph.com: ['type=AVC msg=audit(1650783593.578:7452): avc: denied { ioctl } for pid=49931 comm="iptables" path="/var/lib/containers/storage/overlay/aafdabf96286cbd0c 8143e515d684b0fe214dec80ce8745a515764b1c2aee71b/merged" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1650783593.680:7454): avc: denied { ioctl } for pid=49949 comm="iptables" path="/var/lib/containers/storage/ov erlay/aafdabf96286cbd0c8143e515d684b0fe214dec80ce8745a515764b1c2aee71b/merg ed" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6803238] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803238/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803238/ SELinux denials found on ubuntu@smithi073.front.sepia.ceph.com: ['type=AVC msg=audit(1650782906.382:202): avc: denied { node_bind } for pid=2185 comm="ping" saddr=172.21.15.73 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803254] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:19:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803254/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803254/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4bfd87b0b19448d944a030243315de5 Command failed on smithi105 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28c7908e-c39e-11ec-8c39-001a4aab830c -e sha1=b42bd433b8c85da129c1c277665f5ddab82dd5fb -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6803259] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:05:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803259/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803259/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7e4f9a79f1c24280a03c405243a360a2 [Errno 2] Cannot find file on the remote 'ubuntu@smithi038.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6803260] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:05:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803260/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803260/ SELinux denials found on ubuntu@smithi124.front.sepia.ceph.com: ['type=AVC msg=audit(1650784428.391:200): avc: denied { node_bind } for pid=2114 comm="ping" saddr=172.21.15.124 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803273] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} ----------------------------------------------------------------- time: 00:23:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803273/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803273/ SELinux denials found on ubuntu@smithi163.front.sepia.ceph.com: ['type=AVC msg=audit(1650785214.268:200): avc: denied { node_bind } for pid=2183 comm="ping" saddr=172.21.15.163 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803289] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:09:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803289/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803289/ SELinux denials found on ubuntu@smithi135.front.sepia.ceph.com: ['type=AVC msg=audit(1650785655.166:199): avc: denied { node_bind } for pid=2098 comm="ping" saddr=172.21.15.135 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803298] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:12:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803298/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803298/ SELinux denials found on ubuntu@smithi007.front.sepia.ceph.com: ['type=AVC msg=audit(1650786060.345:200): avc: denied { node_bind } for pid=1945 comm="ping" saddr=172.21.15.7 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803304] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:12:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803304/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803304/ SELinux denials found on ubuntu@smithi116.front.sepia.ceph.com: ['type=AVC msg=audit(1650786540.241:200): avc: denied { node_bind } for pid=2102 comm="ping" saddr=172.21.15.116 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803311] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:13:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803311/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803311/ SELinux denials found on ubuntu@smithi003.front.sepia.ceph.com: ['type=AVC msg=audit(1650786862.626:203): avc: denied { node_bind } for pid=1927 comm="ping" saddr=172.21.15.3 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803321] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:18:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803321/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803321/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6c9ca5db994a483ba608f7457b2f9df0 Command failed on smithi123 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5134361a-c3a5-11ec-8c39-001a4aab830c -e sha1=b42bd433b8c85da129c1c277665f5ddab82dd5fb -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6803327] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/insights} ----------------------------------------------------------------- time: 00:11:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803327/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803327/ SELinux denials found on ubuntu@smithi114.front.sepia.ceph.com: ['type=AVC msg=audit(1650787645.637:200): avc: denied { node_bind } for pid=2074 comm="ping" saddr=172.21.15.114 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803336] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803336/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803336/ SELinux denials found on ubuntu@smithi142.front.sepia.ceph.com: ['type=AVC msg=audit(1650788098.557:200): avc: denied { node_bind } for pid=2118 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803340] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} ----------------------------------------------------------------- time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803340/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803340/ SELinux denials found on ubuntu@smithi099.front.sepia.ceph.com: ['type=AVC msg=audit(1650788324.805:199): avc: denied { node_bind } for pid=2052 comm="ping" saddr=172.21.15.99 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803347] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:13:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803347/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803347/ SELinux denials found on ubuntu@smithi101.front.sepia.ceph.com: ['type=AVC msg=audit(1650788751.830:202): avc: denied { node_bind } for pid=2089 comm="ping" saddr=172.21.15.101 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1650789462.507:7425): avc: denied { ioctl } for pid=50043 comm="iptables" path="/var/lib/containers/storage/overlay/8c6cccf 5bcb34c4f4f7e2a26226bb59c7fa8f405043819a47e73a3ba921088cd/merged" dev="overlay" ino=3539091 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6803353] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:12:31 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803353/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803353/ SELinux denials found on ubuntu@smithi123.front.sepia.ceph.com: ['type=AVC msg=audit(1650789024.283:199): avc: denied { node_bind } for pid=2020 comm="ping" saddr=172.21.15.123 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803357] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:35:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803357/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803357/ SELinux denials found on ubuntu@smithi039.front.sepia.ceph.com: ['type=AVC msg=audit(1650789136.538:200): avc: denied { node_bind } for pid=2049 comm="ping" saddr=172.21.15.39 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803371] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:05:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803371/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803371/ SELinux denials found on ubuntu@smithi083.front.sepia.ceph.com: ['type=AVC msg=audit(1650789444.678:201): avc: denied { node_bind } for pid=2097 comm="ping" saddr=172.21.15.83 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803372] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:13:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803372/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803372/ sentry: https://sentry.ceph.com/organizations/ceph/?query=31735bfd88b44e619b7befffeaf3866f Command failed on smithi002 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:b42bd433b8c85da129c1c277665f5ddab82dd5fb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88647a0a-c3aa-11ec-8c39-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [6803390] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:33:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803390/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803390/ SELinux denials found on ubuntu@smithi114.front.sepia.ceph.com: ['type=AVC msg=audit(1650790402.533:200): avc: denied { node_bind } for pid=2050 comm="ping" saddr=172.21.15.114 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803391] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:09:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803391/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803391/ SELinux denials found on ubuntu@smithi083.front.sepia.ceph.com: ['type=AVC msg=audit(1650790393.899:201): avc: denied { node_bind } for pid=2070 comm="ping" saddr=172.21.15.83 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803398] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:08:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803398/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803398/ SELinux denials found on ubuntu@smithi176.front.sepia.ceph.com: ['type=AVC msg=audit(1650790832.720:200): avc: denied { node_bind } for pid=2036 comm="ping" saddr=172.21.15.176 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803408] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803408/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803408/ SELinux denials found on ubuntu@smithi002.front.sepia.ceph.com: ['type=AVC msg=audit(1650791062.945:204): avc: denied { node_bind } for pid=1941 comm="ping" saddr=172.21.15.2 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803429] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:14:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803429/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803429/ SELinux denials found on ubuntu@smithi145.front.sepia.ceph.com: ['type=AVC msg=audit(1650791799.314:200): avc: denied { node_bind } for pid=2030 comm="ping" saddr=172.21.15.145 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803430] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:09:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803430/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803430/ SELinux denials found on ubuntu@smithi033.front.sepia.ceph.com: ['type=AVC msg=audit(1650791781.439:200): avc: denied { node_bind } for pid=1951 comm="ping" saddr=172.21.15.33 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803433] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:24:48 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803433/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803433/ SELinux denials found on ubuntu@smithi085.front.sepia.ceph.com: ['type=AVC msg=audit(1650792072.468:201): avc: denied { node_bind } for pid=2169 comm="ping" saddr=172.21.15.85 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803436] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:11:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803436/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803436/ SELinux denials found on ubuntu@smithi143.front.sepia.ceph.com: ['type=AVC msg=audit(1650792256.360:200): avc: denied { node_bind } for pid=2008 comm="ping" saddr=172.21.15.143 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803437] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:20:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803437/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803437/ SELinux denials found on ubuntu@smithi007.front.sepia.ceph.com: ['type=AVC msg=audit(1650792238.678:200): avc: denied { node_bind } for pid=1924 comm="ping" saddr=172.21.15.7 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803444] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803444/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803444/ SELinux denials found on ubuntu@smithi003.front.sepia.ceph.com: ['type=AVC msg=audit(1650793308.176:7457): avc: denied { ioctl } for pid=50206 comm="iptables" path="/var/lib/containers/storage/overlay/459f4154bcb73059d 217bdda508e23ebebc0dd2007f2e7413dee6e7cb6f87863/merged" dev="overlay" ino=3539055 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6803461] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:30:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803461/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803461/ SELinux denials found on ubuntu@smithi035.front.sepia.ceph.com: ['type=AVC msg=audit(1650793028.983:200): avc: denied { node_bind } for pid=2041 comm="ping" saddr=172.21.15.35 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803468] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:05:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803468/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803468/ SELinux denials found on ubuntu@smithi106.front.sepia.ceph.com: ['type=AVC msg=audit(1650793309.420:201): avc: denied { node_bind } for pid=2140 comm="ping" saddr=172.21.15.106 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803490] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:20:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803490/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803490/ SELinux denials found on ubuntu@smithi165.front.sepia.ceph.com: ['type=AVC msg=audit(1650794012.596:200): avc: denied { node_bind } for pid=2007 comm="ping" saddr=172.21.15.165 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803498] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:28:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803498/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803498/ SELinux denials found on ubuntu@smithi143.front.sepia.ceph.com: ['type=AVC msg=audit(1650794565.487:204): avc: denied { node_bind } for pid=2000 comm="ping" saddr=172.21.15.143 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803526] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:18:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803526/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803526/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e4d0d0042fcd43eca6f2f9804561733b Command failed on smithi139 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9662f5e2-c3b8-11ec-8c39-001a4aab830c -e sha1=b42bd433b8c85da129c1c277665f5ddab82dd5fb -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6803554] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803554/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803554/ SELinux denials found on ubuntu@smithi143.front.sepia.ceph.com: ['type=AVC msg=audit(1650796783.876:200): avc: denied { node_bind } for pid=2095 comm="ping" saddr=172.21.15.143 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803555] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:35:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803555/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803555/ "2022-04-24T10:45:43.564172+0000 mon.a (mon.0) 10 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log [6803561] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:07:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803561/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803561/ SELinux denials found on ubuntu@smithi173.front.sepia.ceph.com: ['type=AVC msg=audit(1650796994.453:199): avc: denied { node_bind } for pid=2043 comm="ping" saddr=172.21.15.173 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803570] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} ----------------------------------------------------------------- time: 00:19:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803570/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803570/ SELinux denials found on ubuntu@smithi196.front.sepia.ceph.com: ['type=AVC msg=audit(1650797273.568:199): avc: denied { node_bind } for pid=1991 comm="ping" saddr=172.21.15.196 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803573] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/failover} ----------------------------------------------------------------- time: 00:11:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803573/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803573/ SELinux denials found on ubuntu@smithi125.front.sepia.ceph.com: ['type=AVC msg=audit(1650797421.154:200): avc: denied { node_bind } for pid=2053 comm="ping" saddr=172.21.15.125 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803575] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:07:53 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803575/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803575/ SELinux denials found on ubuntu@smithi187.front.sepia.ceph.com: ['type=AVC msg=audit(1650797429.543:200): avc: denied { node_bind } for pid=2017 comm="ping" saddr=172.21.15.187 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803577] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:10:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803577/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803577/ SELinux denials found on ubuntu@smithi099.front.sepia.ceph.com: ['type=AVC msg=audit(1650797457.280:200): avc: denied { node_bind } for pid=1995 comm="ping" saddr=172.21.15.99 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803589] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:28:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803589/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803589/ SELinux denials found on ubuntu@smithi123.front.sepia.ceph.com: ['type=AVC msg=audit(1650798024.456:200): avc: denied { node_bind } for pid=2045 comm="ping" saddr=172.21.15.123 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803596] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:09:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803596/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803596/ SELinux denials found on ubuntu@smithi143.front.sepia.ceph.com: ['type=AVC msg=audit(1650798192.496:199): avc: denied { node_bind } for pid=2041 comm="ping" saddr=172.21.15.143 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6803616] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 01:15:26 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803616/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803616/ SELinux denials found on ubuntu@smithi114.front.sepia.ceph.com: ['type=AVC msg=audit(1650799132.610:200): avc: denied { node_bind } for pid=2066 comm="ping" saddr=172.21.15.114 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] Dead ================================================================= [6803241] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803241/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803241/ sentry: https://sentry.ceph.com/organizations/ceph/?query=178424e8bef7406a925cc2e75f8f3519 SSH connection to smithi080 was lost: 'uname -r' [6803251] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803251/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803251/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ddcf25565c824724b75225b764bf67f6 SSH connection to smithi190 was lost: 'uname -r' [6803295] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803295/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803295/ hit max job timeout [6803379] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803379/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803379/ hit max job timeout [6803384] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803384/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803384/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b18c959495834e7c87c5c6b986933ba2 SSH connection to smithi124 was lost: 'uname -r' [6803434] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803434/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803434/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b66d3d0c72e446f8e7ba68067f9c7a7 SSH connection to smithi115 was lost: 'uname -r' [6803471] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803471/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803471/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ffd90e0850854bc1916acc4ba9f3f82c SSH connection to smithi143 was lost: 'uname -r' [6803493] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803493/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803493/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7463cb689ec34c3cb5388bbf8122f4a9 SSH connection to smithi124 was lost: 'uname -r' [6803519] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803519/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803519/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3e917e01fb3b46c4a80f17e4e997d3e6 SSH connection to smithi006 was lost: 'uname -r' [6803548] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:03:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803548/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803548/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5ce56516f74493ab82ef33fb7662b6f {'smithi039.front.sepia.ceph.com': {'_ansible_no_log': False, 'attempts': 24, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'only_upgrade': False, 'package': None, 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': True, 'upgrade': None}}, 'msg': 'Failed to update apt cache: '}} [6803549] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803549/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803549/ sentry: https://sentry.ceph.com/organizations/ceph/?query=81e0d50db55143e48e98f0297d9024e6 SSH connection to smithi163 was lost: 'uname -r' [6803566] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803566/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803566/ hit max job timeout [6803598] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803598/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803598/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c3ce161f88a4e75b77f6a155c0f4d68 SSH connection to smithi189 was lost: 'uname -r' Pass ================================================================= [6803205] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803205/ [6803206] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803206/ [6803207] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:27:29 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803207/ [6803208] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803208/ [6803209] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:33:53 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803209/ [6803210] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:07:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803210/ [6803211] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:31:36 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803211/ [6803212] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:26:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803212/ [6803213] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:24:04 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803213/ [6803214] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803214/ [6803215] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:31:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803215/ [6803216] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:03:41 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803216/ [6803217] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:13:15 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803217/ [6803218] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 01:14:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803218/ [6803219] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:10:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803219/ [6803220] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:37:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803220/ [6803221] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:20:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803221/ [6803222] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:31 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803222/ [6803223] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:30:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803223/ [6803224] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:30:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803224/ [6803225] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:09:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803225/ [6803226] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803226/ [6803227] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:17:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803227/ [6803228] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803228/ [6803229] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803229/ [6803230] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:34:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803230/ [6803231] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:11:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803231/ [6803232] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803232/ [6803233] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:17:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803233/ [6803234] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:18:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803234/ [6803235] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803235/ [6803236] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 01:00:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803236/ [6803239] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803239/ [6803240] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803240/ [6803242] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:20:18 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803242/ [6803243] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803243/ [6803244] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803244/ [6803245] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:40:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803245/ [6803246] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:31:46 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803246/ [6803247] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:18:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803247/ [6803248] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:35:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803248/ [6803249] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:08:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803249/ [6803250] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803250/ [6803252] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803252/ [6803253] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:28:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803253/ [6803255] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:22:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803255/ [6803256] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:16:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803256/ [6803257] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:28:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803257/ [6803258] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:27:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803258/ [6803261] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:06:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803261/ [6803262] rados/objectstore/{backends/objectstore supported-random-distro$/{centos_8}} time: 04:42:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803262/ [6803263] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803263/ [6803264] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:20:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803264/ [6803265] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803265/ [6803266] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:20:13 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803266/ [6803267] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803267/ [6803268] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803268/ [6803269] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803269/ [6803270] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803270/ [6803271] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803271/ [6803272] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:19:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803272/ [6803274] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803274/ [6803275] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803275/ [6803276] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:22:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803276/ [6803277] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:06:18 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803277/ [6803278] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:23:34 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803278/ [6803279] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:09:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803279/ [6803280] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:37:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803280/ [6803281] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:52:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803281/ [6803282] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:30:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803282/ [6803283] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:14:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803283/ [6803284] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:26:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803284/ [6803285] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/failover} time: 00:21:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803285/ [6803286] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803286/ [6803287] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803287/ [6803288] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:38:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803288/ [6803290] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803290/ [6803291] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803291/ [6803292] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:22:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803292/ [6803293] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803293/ [6803294] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803294/ [6803296] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:01:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803296/ [6803297] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} time: 00:23:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803297/ [6803299] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:19:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803299/ [6803300] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:19:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803300/ [6803301] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:21:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803301/ [6803302] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803302/ [6803303] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:06:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803303/ [6803305] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803305/ [6803306] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:10:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803306/ [6803307] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} time: 02:49:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803307/ [6803308] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803308/ [6803309] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:09:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803309/ [6803310] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:34:03 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803310/ [6803312] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:28:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803312/ [6803313] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:39:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803313/ [6803314] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803314/ [6803315] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:32:41 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803315/ [6803316] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803316/ [6803317] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:19:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803317/ [6803318] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803318/ [6803319] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:34 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803319/ [6803320] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803320/ [6803322] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache} time: 00:19:26 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803322/ [6803323] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr orchestrator_cli} time: 00:22:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803323/ [6803324] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803324/ [6803325] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:18:26 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803325/ [6803326] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:35:32 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803326/ [6803328] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:32 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803328/ [6803329] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803329/ [6803330] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/basic 3-final} time: 00:12:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803330/ [6803331] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803331/ [6803332] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:53 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803332/ [6803333] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803333/ [6803334] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:06:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803334/ [6803335] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 01:08:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803335/ [6803337] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:08:09 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803337/ [6803338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:36:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803338/ [6803339] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:13 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803339/ [6803341] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803341/ [6803342] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/pool-create-delete} time: 00:25:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803342/ [6803343] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:29:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803343/ [6803344] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:29 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803344/ [6803345] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:47:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803345/ [6803346] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803346/ [6803348] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803348/ [6803349] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:14:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803349/ [6803350] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:26:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803350/ [6803351] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803351/ [6803352] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803352/ [6803354] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803354/ [6803355] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:40:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803355/ [6803356] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:34:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803356/ [6803358] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803358/ [6803359] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:32:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803359/ [6803360] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:03 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803360/ [6803361] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:45:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803361/ [6803362] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803362/ [6803363] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects} time: 00:29:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803363/ [6803364] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:23:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803364/ [6803365] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803365/ [6803366] rados/singleton/{all/peer mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:18:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803366/ [6803367] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:51:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803367/ [6803368] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803368/ [6803369] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803369/ [6803370] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:33:34 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803370/ [6803373] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:15:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803373/ [6803374] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803374/ [6803375] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803375/ [6803376] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:23:46 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803376/ [6803377] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:29:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803377/ [6803378] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:23:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803378/ [6803380] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:36 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803380/ [6803381] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:20:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803381/ [6803382] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:26:53 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803382/ [6803383] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_5925} time: 00:10:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803383/ [6803385] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:23:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803385/ [6803386] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803386/ [6803387] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803387/ [6803388] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} time: 00:30:13 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803388/ [6803389] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803389/ [6803392] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:43:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803392/ [6803393] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:38:03 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803393/ [6803394] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803394/ [6803395] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803395/ [6803396] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:23:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803396/ [6803397] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803397/ [6803399] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:34 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803399/ [6803400] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:08:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803400/ [6803401] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803401/ [6803402] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803402/ [6803403] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803403/ [6803404] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:19:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803404/ [6803405] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:33:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803405/ [6803406] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:32 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803406/ [6803407] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803407/ [6803409] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:10:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803409/ [6803410] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:16:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803410/ [6803411] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:31:15 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803411/ [6803412] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:18:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803412/ [6803413] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:17:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803413/ [6803414] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:48:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803414/ [6803415] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803415/ [6803416] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803416/ [6803417] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:24:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803417/ [6803418] rados/singleton/{all/random-eio mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:32:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803418/ [6803419] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:16:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803419/ [6803420] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803420/ [6803421] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803421/ [6803422] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_api_tests} time: 00:42:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803422/ [6803423] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803423/ [6803424] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:17:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803424/ [6803425] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} time: 00:20:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803425/ [6803426] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/rgw 3-final} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803426/ [6803427] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:23:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803427/ [6803428] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803428/ [6803431] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803431/ [6803432] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803432/ [6803435] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:14:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803435/ [6803438] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803438/ [6803439] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:15:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803439/ [6803440] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803440/ [6803441] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:17:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803441/ [6803442] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:15:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803442/ [6803443] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:21:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803443/ [6803445] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803445/ [6803446] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:21:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803446/ [6803447] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:28:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803447/ [6803448] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803448/ [6803449] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:08:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803449/ [6803450] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:19:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803450/ [6803451] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:06:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803451/ [6803452] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803452/ [6803453] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:29:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803453/ [6803454] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:31:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803454/ [6803455] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:28:52 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803455/ [6803456] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803456/ [6803457] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803457/ [6803458] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803458/ [6803459] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:36:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803459/ [6803460] rados/singleton/{all/test-crash mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:18:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803460/ [6803462] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:29:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803462/ [6803463] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803463/ [6803464] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 00:55:31 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803464/ [6803465] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803465/ [6803466] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:16:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803466/ [6803467] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:30:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803467/ [6803469] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:19:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803469/ [6803470] rados/singleton/{all/test-noautoscale-flag mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803470/ [6803472] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803472/ [6803473] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803473/ [6803474] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:52 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803474/ [6803475] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803475/ [6803476] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803476/ [6803477] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:29:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803477/ [6803478] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:06:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803478/ [6803479] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803479/ [6803480] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:22:38 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803480/ [6803481] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:13:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803481/ [6803482] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803482/ [6803483] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:29:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803483/ [6803484] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:24:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803484/ [6803485] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:34:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803485/ [6803486] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803486/ [6803487] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:37:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803487/ [6803488] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:42:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803488/ [6803489] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:32:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803489/ [6803491] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/workunits} time: 00:08:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803491/ [6803492] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 01:13:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803492/ [6803494] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:31:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803494/ [6803495] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803495/ [6803496] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:13:23 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803496/ [6803497] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:06:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803497/ [6803499] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803499/ [6803500] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803500/ [6803501] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:29:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803501/ [6803502] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:18:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803502/ [6803503] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:32:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803503/ [6803504] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/rados_mon_workunits} time: 00:26:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803504/ [6803505] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:07:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803505/ [6803506] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803506/ [6803507] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:06:31 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803507/ [6803508] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803508/ [6803509] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:15 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803509/ [6803510] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:32:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803510/ [6803511] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803511/ [6803512] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:11:59 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803512/ [6803513] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{centos_8} tasks/dashboard} time: 00:58:03 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803513/ [6803514] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:26:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803514/ [6803515] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:48:56 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803515/ [6803516] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:32:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803516/ [6803517] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803517/ [6803518] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:41:00 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803518/ [6803520] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:43:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803520/ [6803521] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:19:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803521/ [6803522] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803522/ [6803523] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:34:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803523/ [6803524] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:39:33 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803524/ [6803525] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803525/ [6803527] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr orchestrator_cli} time: 00:22:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803527/ [6803528] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:37 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803528/ [6803529] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-radosbench} time: 00:49:54 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803529/ [6803530] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:32:08 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803530/ [6803531] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:04 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803531/ [6803532] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803532/ [6803533] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:17:15 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803533/ [6803534] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803534/ [6803535] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:06:16 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803535/ [6803536] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:07:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803536/ [6803537] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:29:36 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803537/ [6803538] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:09 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803538/ [6803539] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:29:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803539/ [6803540] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803540/ [6803541] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:20:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803541/ [6803542] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803542/ [6803543] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind} time: 00:51:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803543/ [6803544] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 00:42:58 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803544/ [6803545] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803545/ [6803546] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803546/ [6803547] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803547/ [6803550] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:04 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803550/ [6803551] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:17:57 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803551/ [6803552] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:29:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803552/ [6803553] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:39:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803553/ [6803556] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803556/ [6803557] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:42:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803557/ [6803558] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:38:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803558/ [6803559] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:29 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803559/ [6803560] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:48:45 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803560/ [6803562] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803562/ [6803563] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:31:07 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803563/ [6803564] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:06:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803564/ [6803565] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803565/ [6803567] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:38:10 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803567/ [6803568] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:28:26 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803568/ [6803569] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803569/ [6803571] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:15:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803571/ [6803572] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803572/ [6803574] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:16:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803574/ [6803576] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:05:24 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803576/ [6803578] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803578/ [6803579] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:05:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803579/ [6803580] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:34 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803580/ [6803581] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803581/ [6803582] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803582/ [6803583] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:19:43 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803583/ [6803584] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:23:42 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803584/ [6803585] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/lockdep} time: 00:15:27 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803585/ [6803586] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:47:05 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803586/ [6803587] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:17 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803587/ [6803588] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:18:49 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803588/ [6803590] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803590/ [6803591] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:23:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803591/ [6803592] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803592/ [6803593] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803593/ [6803594] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:29:20 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803594/ [6803595] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:28:51 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803595/ [6803597] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803597/ [6803599] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:50 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803599/ [6803600] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:13 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803600/ [6803601] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:27:55 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803601/ [6803602] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803602/ [6803603] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} time: 03:10:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803603/ [6803604] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:24:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803604/ [6803605] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:21 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803605/ [6803606] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 01:29:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803606/ [6803607] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:16:02 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803607/ [6803608] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:30 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803608/ [6803609] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:41 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803609/ [6803610] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:21:28 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803610/ [6803611] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:26:39 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803611/ [6803612] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:20:19 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803612/ [6803613] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:18:25 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803613/ [6803614] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/insights} time: 00:20:44 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803614/ [6803615] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803615/ [6803617] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2022-04-24_03:31:03-rados-pacific-distro-default-smithi/6803617/