2022-07-17T03:33:25.576 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-17T03:33:25.577 INFO:teuthology.results:subset: '10133/99999' 2022-07-17T03:33:25.578 INFO:teuthology.results:seed: '1126' 2022-07-17T08:22:51.540 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-17T08:22:51.592 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-17T11:38:14.388 INFO:teuthology.results:Tests finished! gathering results... 2022-07-17T11:38:48.518 INFO:scrape:Found 413 jobs 2022-07-17T11:38:49.275 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933521/teuthology.log 2022-07-17T11:38:51.977 INFO:scrape:Found 8 distinct failure reasons 2022-07-17T11:38:51.979 INFO:scrape:Failure: Command failed on smithi197 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 015c21d8-05a0-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2022-07-17T11:38:51.979 INFO:scrape:5 jobs: ['6933414', '6933276', '6933478', '6933209', '6933350'] 2022-07-17T11:38:51.980 INFO:scrape:suites intersection: [] 2022-07-17T11:38:51.980 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '2-repo_digest/repo_digest', '3-upgrade-with-workload', '3-upgrade/staggered', '4-final}', '4-wait', '5-upgrade-ls', 'mon_election/classic}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04'] 2022-07-17T11:38:51.980 INFO:scrape: 2022-07-17T11:38:51.981 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi093.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2022-07-17T11:38:51.981 INFO:scrape:2 jobs: ['6933499', '6933238'] 2022-07-17T11:38:51.982 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rook/master}'] 2022-07-17T11:38:51.982 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rook/master}'] 2022-07-17T11:38:51.983 INFO:scrape: 2022-07-17T11:38:51.983 INFO:scrape:Failure: Cannot connect to remote host smithi115 2022-07-17T11:38:51.983 INFO:scrape:7 jobs: ['6933404', '6933228', '6933267', '6933411', '6933518', '6933542', '6933264'] 2022-07-17T11:38:51.984 INFO:scrape:suites intersection: [] 2022-07-17T11:38:51.984 INFO:scrape:suites union: ['0-nvme-loop', '1-install/luminous', '1-install/nautilus-v1only', '1-start', '2-services/client-keyring', '2-services/mirror', '3-final}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'distro/ubuntu_18.04', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'rados', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/with-work/{0-distro/ubuntu_18.04', 'start', 'start}', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}', 'workloads/rbd_cls}'] 2022-07-17T11:38:51.985 INFO:scrape: 2022-07-17T11:38:51.985 INFO:scrape:Failure: Command failed on smithi036 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:dea9a88ad6b33408ea98bfc388241b5093ab9cad shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ddb17f22-0599-11ed-842e-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-07-17T11:38:51.986 INFO:scrape:1 jobs: ['6933328'] 2022-07-17T11:38:51.986 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-wait}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2022-07-17T11:38:51.987 INFO:scrape: 2022-07-17T11:38:51.987 INFO:scrape:Dead: None 2022-07-17T11:38:51.988 INFO:scrape:1 jobs: ['6933521'] 2022-07-17T11:38:51.988 INFO:scrape: 2022-07-17T11:38:51.989 INFO:scrape:Failure: 'check osd count' reached maximum tries (90) after waiting for 900 seconds 2022-07-17T11:38:51.990 INFO:scrape:1 jobs: ['6933368'] 2022-07-17T11:38:51.991 INFO:scrape:suites: ['0-kubeadm', '1-rook', '2-workload/radosbench', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.6.2}'] 2022-07-17T11:38:51.991 INFO:scrape: 2022-07-17T11:38:51.992 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi146.front.sepia.ceph.com: ['type=AVC msg=audit(1658045294.716:18208): avc: denied { ioctl } for pid=120309 comm="iptables" path="/var/lib/containers/storage/overlay/0a8cda6a57def8678084f38c5c5d0fa1243b1301b74e28b6cc376f19e9e651ee/merged" dev="overlay" ino=3805348 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-07-17T11:38:51.993 INFO:scrape:3 jobs: ['6933505', '6933297', '6933397'] 2022-07-17T11:38:51.994 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-07-17T11:38:51.995 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-07-17T11:38:51.995 INFO:scrape: 2022-07-17T11:38:51.996 INFO:scrape:Failure: "2022-07-17T03:51:45.569363+0000 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log 2022-07-17T11:38:51.997 INFO:scrape:2 jobs: ['6933149', '6933412'] 2022-07-17T11:38:51.997 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'openstack}', 'rados', 'thrashosds-health'] 2022-07-17T11:38:51.998 INFO:scrape:suites union: ['cluster/{12-osds', 'clusters/{fixed-4', 'mon_election/connectivity', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-shec/{ceph', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashers/fastread', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}'] 2022-07-17T11:38:51.999 INFO:scrape: 2022-07-17T11:38:52.013 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/ failed: 21 dead: 1 running: 0 waiting: 0 queued: 0 passed: 391 Fail ================================================================= [6933149] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933149/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933149/ "2022-07-17T03:51:45.569363+0000 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [6933209] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:22:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933209/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933209/ sentry: https://sentry.ceph.com/organizations/ceph/?query=966cebc02b5d45368335c9f94909ed53 Command failed on smithi115 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fda1300a-0587-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6933228] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933228/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933228/ sentry: https://sentry.ceph.com/organizations/ceph/?query=560ab766639f4d6f83e54fcdf0d06fb9 Cannot connect to remote host smithi204 [6933238] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:07:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933238/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933238/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ce4f8421718f45d0aa3d223f2704fff5 [Errno 2] Cannot find file on the remote 'ubuntu@smithi089.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6933264] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933264/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933264/ sentry: https://sentry.ceph.com/organizations/ceph/?query=067f4a68234a4fafa6bb7445ae775455 Cannot connect to remote host smithi043 [6933267] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933267/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933267/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e93ca3947ab442eb8cd585789b197227 Cannot connect to remote host smithi188 [6933276] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:23:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933276/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933276/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8cd1f902ac4d44aea032717e7b1092a9 Command failed on smithi115 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 613bf8c6-0596-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6933297] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:17:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933297/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933297/ SELinux denials found on ubuntu@smithi022.front.sepia.ceph.com: ['type=AVC msg=audit(1658039160.831:18213): avc: denied { ioctl } for pid=120613 comm="iptables" path="/var/lib/containers/storage/overlay/b0fbf17adcf8defb7 1a2d777a82b12a0f450eb03dea7c4768cb9872c3cae09ce/merged" dev="overlay" ino=3805356 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6933328] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:16:40 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933328/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933328/ sentry: https://sentry.ceph.com/organizations/ceph/?query=08ce8329511e4617982edc81aa4b11f2 Command failed on smithi036 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:dea9a88ad6b33408ea98bfc388241b5093ab9cad shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ddb17f22-0599-11ed-842e-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [6933350] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} ----------------------------------------------------------------- time: 01:01:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933350/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933350/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d069acebd10745b0967dc40f41fcc931 Command failed on smithi043 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 621e9d5c-059b-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.r.z'" [6933368] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933368/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933368/ sentry: https://sentry.ceph.com/organizations/ceph/?query=64f2f4e1e3f24d5dbf743927d538673d 'check osd count' reached maximum tries (90) after waiting for 900 seconds [6933397] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:17:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933397/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933397/ SELinux denials found on ubuntu@smithi075.front.sepia.ceph.com: ['type=AVC msg=audit(1658041938.707:18210): avc: denied { ioctl } for pid=120278 comm="iptables" path="/var/lib/containers/storage/overlay/0346d5eb55687a096 8b4e818f93a1938ddcc35c0fe45597010abbb38ee6dda32/merged" dev="overlay" ino=3805298 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6933404] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933404/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933404/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c6f4ef7973940dcabdfc45940d18cc9 Cannot connect to remote host smithi115 [6933411] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933411/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933411/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6697442a51e84733a50e207d3327cb7a Cannot connect to remote host smithi099 [6933412] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:31:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933412/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933412/ "2022-07-17T07:17:51.137322+0000 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [6933414] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:22:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933414/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933414/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5f45e7191a95461bb2df69cc43a20f6c Command failed on smithi197 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 015c21d8-05a0-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6933478] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:23:13 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933478/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933478/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a1945053951a48b285bcfd6d6ae0af90 Command failed on smithi047 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a054e686-05a4-11ed-842e-001a4aab830c -e sha1=dea9a88ad6b33408ea98bfc388241b5093ab9cad -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6933499] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:07:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933499/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933499/ sentry: https://sentry.ceph.com/organizations/ceph/?query=04a230f012f54183920b57ec33d0256e [Errno 2] Cannot find file on the remote 'ubuntu@smithi093.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6933505] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933505/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933505/ SELinux denials found on ubuntu@smithi146.front.sepia.ceph.com: ['type=AVC msg=audit(1658045294.716:18208): avc: denied { ioctl } for pid=120309 comm="iptables" path="/var/lib/containers/storage/overlay/0a8cda6a57def8678 084f38c5c5d0fa1243b1301b74e28b6cc376f19e9e651ee/merged" dev="overlay" ino=3805348 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6933518] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933518/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933518/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ebe1971ee93e426d801a54db1640a5de Cannot connect to remote host smithi135 [6933542] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933542/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933542/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b45f3cceda7b4af6861224d39beaac3e Cannot connect to remote host smithi114 Dead ================================================================= [6933521] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933521/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933521/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds Pass ================================================================= [6933141] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933141/ [6933142] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:19:45 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933142/ [6933143] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933143/ [6933144] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:59:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933144/ [6933145] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:33:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933145/ [6933146] rados/multimon/{clusters/9 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933146/ [6933147] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933147/ [6933148] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:21:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933148/ [6933150] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933150/ [6933151] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:23:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933151/ [6933152] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:15:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933152/ [6933153] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:10:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933153/ [6933154] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_api_tests} time: 00:28:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933154/ [6933155] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:22:48 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933155/ [6933156] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 02:04:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933156/ [6933157] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:16:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933157/ [6933158] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933158/ [6933159] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:33:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933159/ [6933160] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:48:07 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933160/ [6933161] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:35:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933161/ [6933162] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:29:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933162/ [6933163] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:20:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933163/ [6933164] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933164/ [6933165] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933165/ [6933166] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933166/ [6933167] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:23 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933167/ [6933168] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:19:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933168/ [6933169] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933169/ [6933170] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933170/ [6933171] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933171/ [6933172] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:24:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933172/ [6933173] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:11:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933173/ [6933174] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:07:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933174/ [6933175] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:34:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933175/ [6933176] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933176/ [6933177] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933177/ [6933178] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933178/ [6933179] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:38:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933179/ [6933180] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:31:26 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933180/ [6933181] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:33:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933181/ [6933182] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933182/ [6933183] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933183/ [6933184] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:25:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933184/ [6933185] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933185/ [6933186] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:08:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933186/ [6933187] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933187/ [6933188] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933188/ [6933189] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933189/ [6933190] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933190/ [6933191] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:38:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933191/ [6933192] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:24:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933192/ [6933193] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933193/ [6933194] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:38:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933194/ [6933195] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:15:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933195/ [6933196] rados/standalone/{supported-random-distro$/{rhel_8} workloads/scrub} time: 01:40:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933196/ [6933197] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933197/ [6933198] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:34:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933198/ [6933199] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:52:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933199/ [6933200] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:19:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933200/ [6933201] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:59:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933201/ [6933202] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933202/ [6933203] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:21:46 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933203/ [6933204] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933204/ [6933205] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933205/ [6933206] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933206/ [6933207] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933207/ [6933208] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:53 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933208/ [6933210] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 04:55:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933210/ [6933211] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:15:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933211/ [6933212] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933212/ [6933213] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:08:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933213/ [6933214] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:20:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933214/ [6933215] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933215/ [6933216] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/progress} time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933216/ [6933217] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933217/ [6933218] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933218/ [6933219] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:35:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933219/ [6933220] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933220/ [6933221] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 03:44:36 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933221/ [6933222] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:13:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933222/ [6933223] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:26:05 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933223/ [6933224] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:35:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933224/ [6933225] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933225/ [6933226] rados/multimon/{clusters/3 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:08:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933226/ [6933227] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:10 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933227/ [6933229] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:30:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933229/ [6933230] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:22:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933230/ [6933231] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:28:13 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933231/ [6933232] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:27:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933232/ [6933233] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933233/ [6933234] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933234/ [6933235] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} time: 00:30:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933235/ [6933236] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933236/ [6933237] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933237/ [6933239] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933239/ [6933240] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933240/ [6933241] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:31:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933241/ [6933242] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:36:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933242/ [6933243] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933243/ [6933244] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} time: 00:44:40 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933244/ [6933245] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933245/ [6933246] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933246/ [6933247] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:22:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933247/ [6933248] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933248/ [6933249] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933249/ [6933250] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933250/ [6933251] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933251/ [6933252] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933252/ [6933253] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_8} tasks/dashboard} time: 01:28:47 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933253/ [6933254] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933254/ [6933255] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933255/ [6933256] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933256/ [6933257] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:12:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933257/ [6933258] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} time: 03:06:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933258/ [6933259] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:25:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933259/ [6933260] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:51:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933260/ [6933261] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:12:23 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933261/ [6933262] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:26:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933262/ [6933263] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933263/ [6933265] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933265/ [6933266] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933266/ [6933268] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:15:13 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933268/ [6933269] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933269/ [6933270] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933270/ [6933271] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933271/ [6933272] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933272/ [6933273] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:22:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933273/ [6933274] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:37:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933274/ [6933275] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933275/ [6933277] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933277/ [6933278] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:29:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933278/ [6933279] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:24:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933279/ [6933280] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933280/ [6933281] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:16:22 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933281/ [6933282] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:24:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933282/ [6933283] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:34:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933283/ [6933284] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:11:05 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933284/ [6933285] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:28:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933285/ [6933286] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933286/ [6933287] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933287/ [6933288] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:19:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933288/ [6933289] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933289/ [6933290] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933290/ [6933291] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:38:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933291/ [6933292] rados/singleton/{all/peer mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:08:40 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933292/ [6933293] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:26 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933293/ [6933294] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933294/ [6933295] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:29:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933295/ [6933296] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933296/ [6933298] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933298/ [6933299] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:37:46 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933299/ [6933300] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_striper} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933300/ [6933301] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/workunits} time: 00:09:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933301/ [6933302] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933302/ [6933303] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933303/ [6933304] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933304/ [6933305] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:16:47 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933305/ [6933306] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933306/ [6933307] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:34:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933307/ [6933308] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:23:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933308/ [6933309] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:36:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933309/ [6933310] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933310/ [6933311] rados/multimon/{clusters/9 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933311/ [6933312] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:33:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933312/ [6933313] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:14:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933313/ [6933314] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933314/ [6933315] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:27:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933315/ [6933316] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:41:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933316/ [6933317] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933317/ [6933318] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933318/ [6933319] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933319/ [6933320] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:19:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933320/ [6933321] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:59:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933321/ [6933322] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:10:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933322/ [6933323] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933323/ [6933324] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:20 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933324/ [6933325] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 01:53:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933325/ [6933326] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:16:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933326/ [6933327] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:24:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933327/ [6933329] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933329/ [6933330] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933330/ [6933331] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933331/ [6933332] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:32:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933332/ [6933333] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:39:48 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933333/ [6933334] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:24:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933334/ [6933335] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933335/ [6933336] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933336/ [6933337] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:28:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933337/ [6933338] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/crash} time: 00:15:32 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933338/ [6933339] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:37:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933339/ [6933340] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:10:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933340/ [6933341] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:20:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933341/ [6933342] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933342/ [6933343] rados/singleton/{all/random-eio mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:24:07 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933343/ [6933344] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933344/ [6933345] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:26:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933345/ [6933346] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:12:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933346/ [6933347] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933347/ [6933348] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933348/ [6933349] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933349/ [6933351] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933351/ [6933352] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933352/ [6933353] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:19:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933353/ [6933354] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933354/ [6933355] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:19:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933355/ [6933356] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933356/ [6933357] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:16:21 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933357/ [6933358] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:16:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933358/ [6933359] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933359/ [6933360] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:48:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933360/ [6933361] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:22:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933361/ [6933362] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:21:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933362/ [6933363] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:42:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933363/ [6933364] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933364/ [6933365] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933365/ [6933366] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:19:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933366/ [6933367] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:36:18 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933367/ [6933369] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:35:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933369/ [6933370] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933370/ [6933371] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933371/ [6933372] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:32:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933372/ [6933373] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933373/ [6933374] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:14:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933374/ [6933375] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933375/ [6933376] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:16:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933376/ [6933377] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933377/ [6933378] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:12:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933378/ [6933379] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933379/ [6933380] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:12:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933380/ [6933381] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:31:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933381/ [6933382] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:14:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933382/ [6933383] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:14:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933383/ [6933384] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:29:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933384/ [6933385] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} time: 01:12:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933385/ [6933386] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:12:23 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933386/ [6933387] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933387/ [6933388] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:23:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933388/ [6933389] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933389/ [6933390] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933390/ [6933391] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:12:15 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933391/ [6933392] rados/multimon/{clusters/3 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:14:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933392/ [6933393] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:32 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933393/ [6933394] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:32:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933394/ [6933395] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:39:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933395/ [6933396] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933396/ [6933398] rados/singleton/{all/test-noautoscale-flag mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:08:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933398/ [6933399] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933399/ [6933400] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:27:07 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933400/ [6933401] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:49:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933401/ [6933402] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:27:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933402/ [6933403] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:21:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933403/ [6933405] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:37:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933405/ [6933406] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:49:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933406/ [6933407] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933407/ [6933408] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:10:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933408/ [6933409] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:26:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933409/ [6933410] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:25:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933410/ [6933413] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933413/ [6933415] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933415/ [6933416] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 00:28:10 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933416/ [6933417] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933417/ [6933418] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 01:20:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933418/ [6933419] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:08:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933419/ [6933420] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:18:32 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933420/ [6933421] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933421/ [6933422] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:55:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933422/ [6933423] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:12:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933423/ [6933424] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933424/ [6933425] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933425/ [6933426] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:30:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933426/ [6933427] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933427/ [6933428] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933428/ [6933429] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933429/ [6933430] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:34:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933430/ [6933431] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933431/ [6933432] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933432/ [6933433] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:32:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933433/ [6933434] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:08:37 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933434/ [6933435] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:08:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933435/ [6933436] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:42:13 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933436/ [6933437] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:21:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933437/ [6933438] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:45:47 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933438/ [6933439] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:20:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933439/ [6933440] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} time: 02:28:26 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933440/ [6933441] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:27:18 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933441/ [6933442] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:47:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933442/ [6933443] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:14:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933443/ [6933444] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:34:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933444/ [6933445] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:14:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933445/ [6933446] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:21:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933446/ [6933447] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933447/ [6933448] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:22:54 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933448/ [6933449] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933449/ [6933450] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} time: 00:54:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933450/ [6933451] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933451/ [6933452] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:28 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933452/ [6933453] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:23:46 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933453/ [6933454] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933454/ [6933455] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:18:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933455/ [6933456] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933456/ [6933457] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:12:58 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933457/ [6933458] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:08:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933458/ [6933459] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933459/ [6933460] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:31:12 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933460/ [6933461] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933461/ [6933462] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:57:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933462/ [6933463] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:38:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933463/ [6933464] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:33:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933464/ [6933465] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933465/ [6933466] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:25:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933466/ [6933467] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933467/ [6933468] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/cache-snaps} time: 00:46:25 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933468/ [6933469] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:35 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933469/ [6933470] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933470/ [6933471] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:53 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933471/ [6933472] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:20:49 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933472/ [6933473] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:21:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933473/ [6933474] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:24:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933474/ [6933475] rados/multimon/{clusters/9 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933475/ [6933476] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933476/ [6933477] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:46 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933477/ [6933479] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933479/ [6933480] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:17:30 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933480/ [6933481] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:16:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933481/ [6933482] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:30:31 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933482/ [6933483] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933483/ [6933484] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:08:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933484/ [6933485] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:16:36 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933485/ [6933486] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:21:36 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933486/ [6933487] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933487/ [6933488] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:30:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933488/ [6933489] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933489/ [6933490] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933490/ [6933491] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933491/ [6933492] rados/singleton/{all/deduptool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933492/ [6933493] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 00:44:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933493/ [6933494] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933494/ [6933495] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:22:08 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933495/ [6933496] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933496/ [6933497] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:48 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933497/ [6933498] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:33:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933498/ [6933500] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933500/ [6933501] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933501/ [6933502] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:24:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933502/ [6933503] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:45 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933503/ [6933504] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933504/ [6933506] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/progress} time: 00:20:43 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933506/ [6933507] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:44 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933507/ [6933508] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:15:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933508/ [6933509] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933509/ [6933510] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:36:29 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933510/ [6933511] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:53 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933511/ [6933512] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:26:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933512/ [6933513] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:01 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933513/ [6933514] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:35:46 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933514/ [6933515] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects} time: 00:19:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933515/ [6933516] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:23:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933516/ [6933517] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:33:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933517/ [6933519] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:13:39 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933519/ [6933520] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:27 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933520/ [6933522] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:10:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933522/ [6933523] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:08:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933523/ [6933524] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933524/ [6933525] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:33:16 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933525/ [6933526] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:32:24 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933526/ [6933527] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:08:50 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933527/ [6933528] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:14:55 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933528/ [6933529] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:20:34 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933529/ [6933530] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:28:32 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933530/ [6933531] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933531/ [6933532] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:13:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933532/ [6933533] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:24:52 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933533/ [6933534] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:59:36 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933534/ [6933535] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:17:11 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933535/ [6933536] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:09 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933536/ [6933537] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:57 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933537/ [6933538] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933538/ [6933539] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:42:19 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933539/ [6933540] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:54:03 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933540/ [6933541] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:25:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933541/ [6933543] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 01:18:17 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933543/ [6933544] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:59 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933544/ [6933545] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/prometheus} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933545/ [6933546] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:39:56 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933546/ [6933547] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933547/ [6933548] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:19:18 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933548/ [6933549] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933549/ [6933550] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:06 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933550/ [6933551] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:27:20 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933551/ [6933552] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:28:38 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933552/ [6933553] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 03:09:26 info: http://pulpito.ceph.com/teuthology-2022-07-17_03:31:03-rados-pacific-distro-default-smithi/6933553/