2021-12-26T03:34:14.693 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-26T03:34:14.696 INFO:teuthology.results:subset: '23017/99999' 2021-12-26T03:34:14.696 INFO:teuthology.results:seed: '3355' 2021-12-26T08:13:13.402 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-26T08:13:13.432 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-12-26T19:59:20.988 INFO:teuthology.results:Tests finished! gathering results... 2021-12-26T19:59:59.317 INFO:scrape:Found 420 jobs 2021-12-26T20:00:06.798 INFO:scrape:Found 9 distinct failure reasons 2021-12-26T20:00:06.800 INFO:scrape:Failure: Command failed on smithi017 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' 2021-12-26T20:00:06.801 INFO:scrape:10 jobs: ['6584469', '6584682', '6584535', '6584702', '6584574', '6584594', '6584488', '6584805', '6584679', '6584780'] 2021-12-26T20:00:06.801 INFO:scrape:suites intersection: [] 2021-12-26T20:00:06.802 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-services/client-keyring', '2-services/mirror', '2-services/nfs', '2-services/nfs-ingress', '3-final}', 'distro/ubuntu_18.04', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'start}'] 2021-12-26T20:00:06.803 INFO:scrape: 2021-12-26T20:00:06.804 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi061.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2021-12-26T20:00:06.805 INFO:scrape:2 jobs: ['6584837', '6584573'] 2021-12-26T20:00:06.805 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '3-final', 'k8s/1.21', 'net/calico', 'rook/master}'] 2021-12-26T20:00:06.806 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-12-26T20:00:06.807 INFO:scrape: 2021-12-26T20:00:06.808 INFO:scrape:Failure: Command failed on smithi017 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa62d9b6-6622-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_4' 2021-12-26T20:00:06.808 INFO:scrape:17 jobs: ['6584825', '6584745', '6584448', '6584442', '6584772', '6584522', '6584430', '6584478', '6584618', '6584562', '6584698', '6584584', '6584729', '6584634', '6584456', '6584632', '6584541'] 2021-12-26T20:00:06.809 INFO:scrape:suites intersection: [] 2021-12-26T20:00:06.810 INFO:scrape:suites union: ['1-install/luminous', '1-install/luminous-v1only', '1-install/mimic', '1-install/nautilus', '1-install/nautilus-v1only', '1-install/nautilus-v2only', '1-install/octopus', '1-start', '2-thrash', '3-tasks/rados_api_tests', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0', 'rados/cephadm/with-work/{0-distro/ubuntu_18.04', 'root}', 'start', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] 2021-12-26T20:00:06.811 INFO:scrape: 2021-12-26T20:00:06.812 INFO:scrape:Failure: SSH connection to smithi102 was lost: 'uname -r' 2021-12-26T20:00:06.812 INFO:scrape:7 jobs: ['6584489', '6584684', '6584809', '6584791', '6584661', '6584828', '6584849'] 2021-12-26T20:00:06.813 INFO:scrape:suites intersection: [] 2021-12-26T20:00:06.814 INFO:scrape:suites union: ['0-nvme-loop', '1-install/luminous', '1-install/mimic-v1only', '1-install/octopus', '1-start', '2-ops/rm-zap-wait}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v2only', 'rados', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/cephadm/with-work/{0-distro/ubuntu_18.04', 'start', 'tasks/rados_python}', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/rbd_cls}', 'workloads/test_rbd_api}'] 2021-12-26T20:00:06.814 INFO:scrape: 2021-12-26T20:00:06.815 INFO:scrape:Dead: b'2021-12-26T18:51:36.112 DEBUG:teuthology.exit:Finished running handlers' 2021-12-26T20:00:06.816 INFO:scrape:10 jobs: ['6584709', '6584806', '6584498', '6584701', '6584566', '6584674', '6584462', '6584603', '6584776', '6584812'] 2021-12-26T20:00:06.817 INFO:scrape:suites intersection: [] 2021-12-26T20:00:06.817 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'cluster/{12-osds', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2021-12-26T20:00:06.818 INFO:scrape: 2021-12-26T20:00:06.819 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e59e5ee06523d706b765968be5a73478a9da3d0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-12-26T20:00:06.820 INFO:scrape:2 jobs: ['6584601', '6584810'] 2021-12-26T20:00:06.820 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/centos_8.3_container_tools_3.0', 'task/test_e2e}'] 2021-12-26T20:00:06.821 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/centos_8.3_container_tools_3.0', 'task/test_e2e}'] 2021-12-26T20:00:06.822 INFO:scrape: 2021-12-26T20:00:06.822 INFO:scrape:Failure: "2021-12-26T06:26:44.072197+0000 osd.7 (osd.7) 62 : cluster [ERR] 3.13s0 scrub : stat mismatch, got 2/1 objects, 0/0 clones, 2/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 13598720/11124736 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log 2021-12-26T20:00:06.823 INFO:scrape:1 jobs: ['6584635'] 2021-12-26T20:00:06.824 INFO:scrape:suites: ['cluster/{12-osds', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-comp-lz4', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2021-12-26T20:00:06.824 INFO:scrape: 2021-12-26T20:00:06.825 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2021-12-26T20:00:06.826 INFO:scrape:1 jobs: ['6584563'] 2021-12-26T20:00:06.827 INFO:scrape:suites: ['clusters/9-mons', 'mon_election/classic', 'msgr-failures/mon-delay', 'msgr/async', 'objectstore/filestore-xfs', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'thrashers/sync-many', 'workloads/rados_api_tests}'] 2021-12-26T20:00:06.827 INFO:scrape: 2021-12-26T20:00:06.828 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/rados/test.sh 2021-12-26T20:00:06.829 INFO:scrape:1 jobs: ['6584721'] 2021-12-26T20:00:06.829 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/lockdep}'] 2021-12-26T20:00:06.830 INFO:scrape: 2021-12-26T20:00:06.849 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/ failed: 34 dead: 17 running: 0 waiting: 0 queued: 0 passed: 369 Fail ================================================================= [6584430] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584430/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584430/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be583cc4022d4b8dbfb0160057d3db24 Command failed on smithi073 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1cb1bd4-65fe-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi073:vg_nvme/lv_4' [6584442] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:15:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584442/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584442/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9c17c9e00009442c991bbc45af94d138 Command failed on smithi117 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d6fe911c-6600-11ec-8c31-001a4aab830c -- ceph orch apply node-exporter '2;smithi115=a;smithi117=b'" [6584448] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584448/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584448/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c9712293d6f4c18aa04ff235b4bd3e4 Command failed on smithi029 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cf68212a-6600-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi029:vg_nvme/lv_4' [6584456] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584456/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584456/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e2fc1e6912a748e88dc597cfc3f54c56 Command failed on smithi063 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c9b90e2-6601-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi063:vg_nvme/lv_4' [6584469] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:03:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584469/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584469/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3b95b653d0f43749f15de8d91abfcb9 Command failed on smithi017 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584478] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584478/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584478/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b9c89c7f9b60440cbb784d5ae03b4f51 Command failed on smithi061 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c2b91f2-6603-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi061:vg_nvme/lv_4' [6584488] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584488/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584488/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c6e567c3c7624be1aeb94818f281c625 Command failed on smithi063 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584522] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:09:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584522/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584522/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ff258e9bb26443a0af6551e8c714ae35 Command failed on smithi094 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a579a6f8-6606-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi094:vg_nvme/lv_4' [6584535] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584535/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584535/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6d629a4dabb345e98a06883c2441d62a Command failed on smithi023 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584541] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584541/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584541/ sentry: https://sentry.ceph.com/organizations/ceph/?query=13e807937a8d4f3295b406966b64ab95 Command failed on smithi062 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e8f83ae-6608-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_4' [6584562] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:09:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584562/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584562/ sentry: https://sentry.ceph.com/organizations/ceph/?query=532c3d4e4ffd458f94976490e7637e45 Command failed on smithi023 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7aa1a4c-6609-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi023:vg_nvme/lv_4' [6584563] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:26:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584563/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584563/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6083c80e96834dd19988106deaae5f64 Command failed (workunit test rados/test.sh) on smithi122 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e59e5ee06523d706b765968be5a73478a9da3d0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [6584573] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:05:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584573/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584573/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d839ba3ff0f64fe8abd1ddceda0baf3b [Errno 2] Cannot find file on the remote 'ubuntu@smithi072.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6584574] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584574/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584574/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69793456e968415a89beb25949dcaa0c Command failed on smithi052 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584584] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584584/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584584/ sentry: https://sentry.ceph.com/organizations/ceph/?query=636855796d4c4c5c971bc9d036ad16cf Command failed on smithi155 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a54a92e-660e-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_4' [6584594] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:03:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584594/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584594/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec3604158fd1488699a83fc85c18a731 Command failed on smithi153 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584601] rados/cephadm/dashboard/{0-distro/centos_8.3_container_tools_3.0 task/test_e2e} ----------------------------------------------------------------- time: 00:19:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584601/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584601/ sentry: https://sentry.ceph.com/organizations/ceph/?query=97336420e0184a3998403c9914c0e808 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e59e5ee06523d706b765968be5a73478a9da3d0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6584618] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584618/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584618/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e6f74fb413624edfb0455d765c285f29 Command failed on smithi038 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8d749c06-6611-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi038:vg_nvme/lv_4' [6584632] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584632/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584632/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bb763c4aef194559a63cab8621a9b7b0 Command failed on smithi049 with status 22: 'sudo cephadm --image quay.ceph.io/ceph-ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8632334e-6612-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi049:vg_nvme/lv_4' [6584634] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:09:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584634/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584634/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0a0530a09b5a4204ac3872c0ca66c53b Command failed on smithi062 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1a0b0ea6-6613-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_4' [6584635] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:28:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584635/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584635/ "2021-12-26T06:26:44.072197+0000 osd.7 (osd.7) 62 : cluster [ERR] 3.13s0 scrub : stat mismatch, got 2/1 objects, 0/0 clones, 2/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 13598720/11124736 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log [6584679] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:03:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584679/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584679/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40a8613a9446485085e1e28ea0df445f Command failed on smithi133 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584682] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:03:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584682/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584682/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fe1bf5a593d45038551fd0220ce56c4 Command failed on smithi039 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584698] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:08:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584698/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c45777dacba4464b9ca7cccf8c3b572f Command failed on smithi060 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid db179920-6618-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi060:vg_nvme/lv_4' [6584702] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:03:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584702/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584702/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a21f8dcd8ab549d4a730b77833c94d24 Command failed on smithi052 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584721] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 06:14:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584721/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584721/ sentry: https://sentry.ceph.com/organizations/ceph/?query=701b1f318963492cb731d7e9fb0db3da Command failed (workunit test rados/test.sh) on smithi137 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e59e5ee06523d706b765968be5a73478a9da3d0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [6584729] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584729/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584729/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e9620036f844230a6a56880b9daec60 Command failed on smithi074 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cc65e056-661a-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi074:vg_nvme/lv_4' [6584745] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:09:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584745/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584745/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d68a54082d0946b68076fec3410c4d7d Command failed on smithi140 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3b3ec898-661c-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi140:vg_nvme/lv_4' [6584772] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584772/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584772/ sentry: https://sentry.ceph.com/organizations/ceph/?query=22bee832e11a4dcd852f58c3d1736f4d Command failed on smithi038 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4188c580-661e-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi038:vg_nvme/lv_4' [6584780] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:03:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584780/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584780/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8413937a1cd474a8c91f07bbb4b45cf Command failed on smithi146 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584805] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584805/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584805/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2c6c07f61b8549e78d586a6a8177609c Command failed on smithi023 with status 251: 'sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn' [6584810] rados/cephadm/dashboard/{0-distro/centos_8.3_container_tools_3.0 task/test_e2e} ----------------------------------------------------------------- time: 00:19:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584810/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584810/ sentry: https://sentry.ceph.com/organizations/ceph/?query=75b537a4970346ee9a7d1764d43060c9 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7e59e5ee06523d706b765968be5a73478a9da3d0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6584825] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:09:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584825/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584825/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b9142192d7e48c3bd9d7459bf47dd79 Command failed on smithi017 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:bb6237f728a28d0a16efe3ac8107d7882d53d03c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa62d9b6-6622-11ec-8c31-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_4' [6584837] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:05:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584837/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584837/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c992b67c4fc64ff3baa78af6441eb83e [Errno 2] Cannot find file on the remote 'ubuntu@smithi061.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' Dead ================================================================= [6584462] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584462/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584462/ hit max job timeout [6584489] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584489/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584489/ sentry: https://sentry.ceph.com/organizations/ceph/?query=78dbb63949b04076ad16c03c578de0bb SSH connection to smithi102 was lost: 'uname -r' [6584498] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584498/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584498/ hit max job timeout [6584566] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584566/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584566/ hit max job timeout [6584603] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584603/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584603/ hit max job timeout [6584661] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584661/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584661/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2cd04ec1c0be4679b5feeb7297b05cc0 SSH connection to smithi152 was lost: 'uname -r' [6584674] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584674/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584674/ hit max job timeout [6584684] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584684/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584684/ sentry: https://sentry.ceph.com/organizations/ceph/?query=429175a2655b43a58ac6ecaa77648e26 SSH connection to smithi125 was lost: 'uname -r' [6584701] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584701/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584701/ hit max job timeout [6584709] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584709/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584709/ hit max job timeout [6584776] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584776/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584776/ hit max job timeout [6584791] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584791/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584791/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a3cba5b9e7a04a0b89460dc1b4ca292e SSH connection to smithi154 was lost: 'uname -r' [6584806] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584806/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584806/ hit max job timeout [6584809] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584809/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584809/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a680d031d6724b4e98d14f04d73f0354 SSH connection to smithi160 was lost: 'uname -r' [6584812] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584812/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584812/ hit max job timeout [6584828] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584828/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584828/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6efcf36f65a24c0fa1430e6d75668afc SSH connection to smithi154 was lost: 'uname -r' [6584849] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584849/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1436ac3318f4eb6bcff1cb4802bfd73 SSH connection to smithi154 was lost: 'uname -r' Pass ================================================================= [6584431] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584431/ [6584432] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:45:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584432/ [6584433] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584433/ [6584434] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:23:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584434/ [6584435] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:33:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584435/ [6584436] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584436/ [6584437] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:16:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584437/ [6584438] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584438/ [6584439] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:51:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584439/ [6584440] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/snaps-few-objects} time: 00:25:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584440/ [6584441] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584441/ [6584443] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:41:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584443/ [6584444] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} time: 00:30:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584444/ [6584445] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:19:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584445/ [6584446] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584446/ [6584447] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584447/ [6584449] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584449/ [6584450] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584450/ [6584451] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:45:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584451/ [6584452] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:22:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584452/ [6584453] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_adoption} time: 00:10:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584453/ [6584454] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584454/ [6584455] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584455/ [6584457] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:21:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584457/ [6584458] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584458/ [6584459] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584459/ [6584460] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584460/ [6584461] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} time: 00:15:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584461/ [6584463] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584463/ [6584464] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584464/ [6584465] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584465/ [6584466] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584466/ [6584467] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584467/ [6584468] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:37:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584468/ [6584470] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584470/ [6584471] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584471/ [6584472] rados/singleton/{all/peer mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584472/ [6584473] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:36:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584473/ [6584474] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:26:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584474/ [6584475] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584475/ [6584476] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:45:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584476/ [6584477] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:20:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584477/ [6584479] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584479/ [6584480] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584480/ [6584481] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584481/ [6584482] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/pool-create-delete} time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584482/ [6584483] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} time: 00:14:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584483/ [6584484] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584484/ [6584485] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:19:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584485/ [6584486] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584486/ [6584487] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:19:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584487/ [6584490] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584490/ [6584491] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:27:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584491/ [6584492] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:21:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584492/ [6584493] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584493/ [6584494] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:23:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584494/ [6584495] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584495/ [6584496] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584496/ [6584497] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:40:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584497/ [6584499] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:12:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584499/ [6584500] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:16:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584500/ [6584501] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584501/ [6584502] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:09:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584502/ [6584503] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:25:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584503/ [6584504] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584504/ [6584505] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:22:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584505/ [6584506] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:51:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584506/ [6584507] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:29:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584507/ [6584508] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:31:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584508/ [6584509] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584509/ [6584510] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584510/ [6584511] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/progress} time: 00:32:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584511/ [6584512] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584512/ [6584513] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:28:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584513/ [6584514] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584514/ [6584515] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm_repos} time: 00:06:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584515/ [6584516] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584516/ [6584517] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584517/ [6584518] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:18:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584518/ [6584519] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:13:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584519/ [6584520] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:21:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584520/ [6584521] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584521/ [6584523] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_5925} time: 00:10:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584523/ [6584524] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584524/ [6584525] rados/singleton/{all/random-eio mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:35:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584525/ [6584526] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:10:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584526/ [6584527] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:34:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584527/ [6584528] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:29:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584528/ [6584529] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584529/ [6584530] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:19:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584530/ [6584531] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584531/ [6584532] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584532/ [6584533] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:27:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584533/ [6584534] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584534/ [6584536] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:25:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584536/ [6584537] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:13:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584537/ [6584538] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584538/ [6584539] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:21:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584539/ [6584540] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584540/ [6584542] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:37:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584542/ [6584543] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:33:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584543/ [6584544] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584544/ [6584545] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584545/ [6584546] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:20:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584546/ [6584547] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:21:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584547/ [6584548] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584548/ [6584549] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584549/ [6584550] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584550/ [6584551] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:22:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584551/ [6584552] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584552/ [6584553] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584553/ [6584554] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584554/ [6584555] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:12:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584555/ [6584556] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:11:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584556/ [6584557] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:14:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584557/ [6584558] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584558/ [6584559] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584559/ [6584560] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} time: 00:41:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584560/ [6584561] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:20:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584561/ [6584564] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:28:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584564/ [6584565] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:42:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584565/ [6584567] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584567/ [6584568] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:28:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584568/ [6584569] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:19:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584569/ [6584570] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584570/ [6584571] rados/singleton/{all/test-crash mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:11:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584571/ [6584572] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_orch_cli} time: 00:13:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584572/ [6584575] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:45:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584575/ [6584576] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584576/ [6584577] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584577/ [6584578] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:34:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584578/ [6584579] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:18:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584579/ [6584580] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584580/ [6584581] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584581/ [6584582] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:23:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584582/ [6584583] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584583/ [6584585] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584585/ [6584586] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:17:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584586/ [6584587] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:42:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584587/ [6584588] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584588/ [6584589] rados/dashboard/{centos_8.2_container_tools_3.0 clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_8} tasks/dashboard} time: 01:11:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584589/ [6584590] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584590/ [6584591] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:43:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584591/ [6584592] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 01:13:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584592/ [6584593] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/workunits} time: 00:20:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584593/ [6584595] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584595/ [6584596] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584596/ [6584597] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:34:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584597/ [6584598] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:23:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584598/ [6584599] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind} time: 00:46:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584599/ [6584600] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:19:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584600/ [6584602] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584602/ [6584604] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:38:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584604/ [6584605] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584605/ [6584606] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.3_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584606/ [6584607] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:35:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584607/ [6584608] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584608/ [6584609] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584609/ [6584610] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:31:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584610/ [6584611] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:06:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584611/ [6584612] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:23:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584612/ [6584613] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:09:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584613/ [6584614] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:18:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584614/ [6584615] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:06:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584615/ [6584616] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_adoption} time: 00:10:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584616/ [6584617] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584617/ [6584619] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:44:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584619/ [6584620] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 00:42:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584620/ [6584621] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584621/ [6584622] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584622/ [6584623] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:27:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584623/ [6584624] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:31:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584624/ [6584625] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-radosbench} time: 00:54:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584625/ [6584626] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584626/ [6584627] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584627/ [6584628] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584628/ [6584629] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:31:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584629/ [6584630] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:33:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584630/ [6584631] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584631/ [6584633] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/crash} time: 00:12:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584633/ [6584636] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:45:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584636/ [6584637] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584637/ [6584638] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:12:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584638/ [6584639] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/lockdep} time: 00:17:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584639/ [6584640] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} time: 00:13:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584640/ [6584641] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584641/ [6584642] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584642/ [6584643] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584643/ [6584644] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584644/ [6584645] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:22:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584645/ [6584646] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:14:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584646/ [6584647] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:08:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584647/ [6584648] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:26:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584648/ [6584649] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:27:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584649/ [6584650] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:31:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584650/ [6584651] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:19:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584651/ [6584652] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584652/ [6584653] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:20:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584653/ [6584654] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:26:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584654/ [6584655] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584655/ [6584656] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584656/ [6584657] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:30:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584657/ [6584658] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:29:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584658/ [6584659] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:26:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584659/ [6584660] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:21:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584660/ [6584662] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:11:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584662/ [6584663] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584663/ [6584664] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584664/ [6584665] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584665/ [6584666] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:32:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584666/ [6584667] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:19:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584667/ [6584668] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584668/ [6584669] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:06:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584669/ [6584670] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:26:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584670/ [6584671] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm_repos} time: 00:07:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584671/ [6584672] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584672/ [6584673] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/failover} time: 00:24:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584673/ [6584675] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584675/ [6584676] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584676/ [6584677] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584677/ [6584678] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584678/ [6584680] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/valgrind} time: 00:28:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584680/ [6584681] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:07:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584681/ [6584683] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 03:16:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584683/ [6584685] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584685/ [6584686] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:34:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584686/ [6584687] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:35:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584687/ [6584688] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584688/ [6584689] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/many workloads/snaps-few-objects} time: 00:35:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584689/ [6584690] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:20:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584690/ [6584691] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:08:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584691/ [6584692] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584692/ [6584693] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:20:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584693/ [6584694] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584694/ [6584695] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584695/ [6584696] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:22:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584696/ [6584697] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584697/ [6584699] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584699/ [6584700] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:08:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584700/ [6584703] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584703/ [6584704] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584704/ [6584705] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:34:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584705/ [6584706] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584706/ [6584707] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:23:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584707/ [6584708] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584708/ [6584710] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584710/ [6584711] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/insights} time: 00:14:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584711/ [6584712] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584712/ [6584713] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:56:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584713/ [6584714] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:22:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584714/ [6584715] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584715/ [6584716] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584716/ [6584717] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584717/ [6584718] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584718/ [6584719] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:41:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584719/ [6584720] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:43:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584720/ [6584722] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:31:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584722/ [6584723] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 01:03:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584723/ [6584724] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584724/ [6584725] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584725/ [6584726] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584726/ [6584727] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:14:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584727/ [6584728] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:25:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584728/ [6584730] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:31:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584730/ [6584731] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584731/ [6584732] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_orch_cli} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584732/ [6584733] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/one workloads/pool-create-delete} time: 00:20:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584733/ [6584734] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584734/ [6584735] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584735/ [6584736] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584736/ [6584737] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584737/ [6584738] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584738/ [6584739] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:33:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584739/ [6584740] rados/standalone/{supported-random-distro$/{rhel_8} workloads/scrub} time: 01:52:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584740/ [6584741] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:12:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584741/ [6584742] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:36:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584742/ [6584743] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:26:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584743/ [6584744] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 01:20:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584744/ [6584746] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584746/ [6584747] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584747/ [6584748] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:51:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584748/ [6584749] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:30:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584749/ [6584750] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:24:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584750/ [6584751] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:40:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584751/ [6584752] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:19:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584752/ [6584753] rados/objectstore/{backends/objectstore supported-random-distro$/{centos_8}} time: 04:39:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584753/ [6584754] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:11:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584754/ [6584755] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 01:35:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584755/ [6584756] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:30:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584756/ [6584757] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:21:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584757/ [6584758] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:34:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584758/ [6584759] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584759/ [6584760] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584760/ [6584761] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584761/ [6584762] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584762/ [6584763] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_cls_all validater/valgrind} time: 01:49:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584763/ [6584764] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:23:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584764/ [6584765] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584765/ [6584766] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584766/ [6584767] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584767/ [6584768] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584768/ [6584769] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584769/ [6584770] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:19:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584770/ [6584771] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:36:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584771/ [6584773] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_5925} time: 00:21:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584773/ [6584774] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584774/ [6584775] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:54:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584775/ [6584777] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:19:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584777/ [6584778] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584778/ [6584779] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:17:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584779/ [6584781] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:47:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584781/ [6584782] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:28:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584782/ [6584783] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:29:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584783/ [6584784] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584784/ [6584785] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:23:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584785/ [6584786] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584786/ [6584787] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:13:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584787/ [6584788] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm} time: 00:15:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584788/ [6584789] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584789/ [6584790] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:19:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584790/ [6584792] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:21:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584792/ [6584793] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:17:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584793/ [6584794] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584794/ [6584795] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:17:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584795/ [6584796] rados/dashboard/{centos_8.2_container_tools_3.0 clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{filestore-xfs} supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584796/ [6584797] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:06:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584797/ [6584798] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584798/ [6584799] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:31:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584799/ [6584800] rados/standalone/{supported-random-distro$/{centos_8} workloads/crush} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584800/ [6584801] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} time: 02:55:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584801/ [6584802] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:20:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584802/ [6584803] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:09:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584803/ [6584804] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:10:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584804/ [6584807] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584807/ [6584808] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/lockdep} time: 00:12:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584808/ [6584811] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584811/ [6584813] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:22:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584813/ [6584814] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584814/ [6584815] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.3_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:14:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584815/ [6584816] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_api_tests} time: 00:28:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584816/ [6584817] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584817/ [6584818] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:14:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584818/ [6584819] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584819/ [6584820] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:09:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584820/ [6584821] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:13:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584821/ [6584822] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:16:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584822/ [6584823] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:29:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584823/ [6584824] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:25:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584824/ [6584826] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:28:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584826/ [6584827] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584827/ [6584829] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:37:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584829/ [6584830] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:38:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584830/ [6584831] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:28:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584831/ [6584832] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584832/ [6584833] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:10:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584833/ [6584834] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584834/ [6584835] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:06:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584835/ [6584836] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584836/ [6584838] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:26:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584838/ [6584839] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584839/ [6584840] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584840/ [6584841] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584841/ [6584842] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:14:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584842/ [6584843] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:37:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584843/ [6584844] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584844/ [6584845] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584845/ [6584846] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:21:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584846/ [6584847] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:46:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584847/ [6584848] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_api_tests validater/valgrind} time: 00:45:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_03:31:03-rados-pacific-distro-default-smithi/6584848/