2021-12-27T05:19:06.191 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-27T05:19:06.193 INFO:teuthology.results:subset: '2010/100000' 2021-12-27T05:19:06.194 INFO:teuthology.results:seed: '8722' 2021-12-27T16:06:38.910 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-27T16:06:38.935 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-12-28T04:06:43.187 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-12-28T04:07:20.050 INFO:scrape:Found 418 jobs 2021-12-28T04:07:20.790 INFO:scrape:Found 4 distinct failure reasons 2021-12-28T04:07:20.792 INFO:scrape:Failure: timeout expired in wait_until_healthy 2021-12-28T04:07:20.792 INFO:scrape:1 jobs: ['6585478'] 2021-12-28T04:07:20.793 INFO:scrape:suites: ['mon_election/connectivity', 'rados/cephadm/workunits/{agent/on', 'task/test_orch_cli}'] 2021-12-28T04:07:20.793 INFO:scrape: 2021-12-28T04:07:20.794 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi168 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=31afabe6f2c6a705bfffc122eab54d273b48481a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-12-28T04:07:20.794 INFO:scrape:2 jobs: ['6585097', '6585307'] 2021-12-28T04:07:20.794 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2021-12-28T04:07:20.795 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-hybrid}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2021-12-28T04:07:20.795 INFO:scrape: 2021-12-28T04:07:20.796 INFO:scrape:Dead: b'2021-12-28T04:04:49.424 DEBUG:teuthology.exit:Finished running handlers' 2021-12-28T04:07:20.796 INFO:scrape:3 jobs: ['6585482', '6585147', '6585351'] 2021-12-28T04:07:20.797 INFO:scrape:suites intersection: [] 2021-12-28T04:07:20.797 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'mon_election/connectivity', 'msgr-failures/none', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'rados/singleton/{all/thrash-eio', 'supported-random-distro$/{centos_8.stream}}'] 2021-12-28T04:07:20.798 INFO:scrape: 2021-12-28T04:07:20.798 INFO:scrape:Failure: Command failed on smithi115 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:31afabe6f2c6a705bfffc122eab54d273b48481a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9cb6894a-66d9-11ec-8c31-001a4aab830c -- ceph mon dump -f json' 2021-12-28T04:07:20.799 INFO:scrape:1 jobs: ['6585130'] 2021-12-28T04:07:20.799 INFO:scrape:suites: ['0-nvme-loop', 'agent/off', 'fixed-2', 'mon_election/connectivity', 'rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0', 'start}'] 2021-12-28T04:07:20.800 INFO:scrape: 2021-12-28T04:07:20.814 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/ failed: 4 dead: 2 running: 1 waiting: 0 queued: 0 passed: 411 Fail ================================================================= [6585097] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585097/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585097/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3cfd29a659934da08c8ab8ee756d2c42 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi168 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=31afabe6f2c6a705bfffc122eab54d273b48481a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6585130] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585130/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585130/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2fe2988fe7d0433c8db6c1d4921bc62f Command failed on smithi115 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:31afabe6f2c6a705bfffc122eab54d273b48481a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9cb6894a-66d9-11ec-8c31-001a4aab830c -- ceph mon dump -f json' [6585307] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} ----------------------------------------------------------------- time: 00:17:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585307/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585307/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5a9a3dde36a64c7ebcf61959d9cbb0fb Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=31afabe6f2c6a705bfffc122eab54d273b48481a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6585478] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:16:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585478/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585478/ sentry: https://sentry.ceph.com/organizations/ceph/?query=23818f820fb74ce18c6a12dd19513af7 timeout expired in wait_until_healthy Dead ================================================================= [6585147] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585147/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585147/ hit max job timeout [6585351] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585351/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585351/ hit max job timeout Running ================================================================= [6585482] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream}} info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585482/ Pass ================================================================= [6585079] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:27:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585079/ [6585080] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585080/ [6585081] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585081/ [6585082] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/host rook/master} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585082/ [6585083] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585083/ [6585084] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:31:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585084/ [6585085] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:12:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585085/ [6585086] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585086/ [6585087] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:53:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585087/ [6585088] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8.stream}} time: 02:08:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585088/ [6585089] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:34:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585089/ [6585090] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:18:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585090/ [6585091] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:14:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585091/ [6585092] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585092/ [6585093] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:31:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585093/ [6585094] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:25:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585094/ [6585095] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585095/ [6585096] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:47:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585096/ [6585098] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585098/ [6585099] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:09:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585099/ [6585100] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:33:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585100/ [6585101] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/repave-all} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585101/ [6585102] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/many workloads/pool-create-delete} time: 00:17:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585102/ [6585103] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} tasks/rados_striper} time: 00:08:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585103/ [6585104] rados/cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm_repos} time: 00:04:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585104/ [6585105] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:43:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585105/ [6585106] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585106/ [6585107] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:21:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585107/ [6585108] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585108/ [6585109] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585109/ [6585110] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:13:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585110/ [6585111] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:09:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585111/ [6585112] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585112/ [6585113] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/progress} time: 00:33:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585113/ [6585114] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/on orchestrator_cli} time: 00:14:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585114/ [6585115] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:23:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585115/ [6585116] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585116/ [6585117] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585117/ [6585118] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:13:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585118/ [6585119] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585119/ [6585120] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585120/ [6585121] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585121/ [6585122] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:36:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585122/ [6585123] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:07:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585123/ [6585124] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:10:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585124/ [6585125] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:24:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585125/ [6585126] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585126/ [6585127] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{rhel_8}} time: 02:30:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585127/ [6585128] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585128/ [6585129] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:19:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585129/ [6585131] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:16:30 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585131/ [6585132] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream}} time: 00:26:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585132/ [6585133] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:15:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585133/ [6585134] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585134/ [6585135] rados/cephadm/workunits/{agent/on mon_election/classic task/test_nfs} time: 00:28:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585135/ [6585136] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} tasks/rados_workunit_loadgen_big} time: 00:29:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585136/ [6585137] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/redirect} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585137/ [6585138] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585138/ [6585139] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:10:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585139/ [6585140] rados/multimon/{clusters/3 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} tasks/mon_recovery} time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585140/ [6585141] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:01:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585141/ [6585142] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_5925} time: 00:08:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585142/ [6585143] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/prometheus} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585143/ [6585144] rados/singleton/{all/deduptool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585144/ [6585145] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:37:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585145/ [6585146] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585146/ [6585148] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:25:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585148/ [6585149] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:31:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585149/ [6585150] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:34:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585150/ [6585151] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585151/ [6585152] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585152/ [6585153] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585153/ [6585154] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585154/ [6585155] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:18:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585155/ [6585156] rados/standalone/{supported-random-distro$/{rhel_8} workloads/scrub} time: 03:10:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585156/ [6585157] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:21:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585157/ [6585158] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585158/ [6585159] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:32:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585159/ [6585160] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585160/ [6585161] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585161/ [6585162] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{rhel_8}} time: 00:41:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585162/ [6585163] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:45:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585163/ [6585164] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:21:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585164/ [6585165] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} tasks/rados_workunit_loadgen_mix} time: 00:18:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585165/ [6585166] rados/cephadm/workunits/{agent/off mon_election/connectivity task/test_orch_cli} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585166/ [6585167] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:08:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585167/ [6585168] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585168/ [6585169] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:38:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585169/ [6585170] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8.stream} tasks/workunits} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585170/ [6585171] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:21:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585171/ [6585172] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585172/ [6585173] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585173/ [6585174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:15:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585174/ [6585175] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:24:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585175/ [6585176] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:08:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585176/ [6585177] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:17:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585177/ [6585178] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:43:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585178/ [6585179] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:14:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585179/ [6585180] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} tasks/mon_clock_no_skews} time: 00:08:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585180/ [6585181] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585181/ [6585182] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_api_tests} time: 00:38:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585182/ [6585183] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585183/ [6585184] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.2} time: 00:16:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585184/ [6585185] rados/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:42:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585185/ [6585186] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 01:10:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585186/ [6585187] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585187/ [6585188] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585188/ [6585189] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585189/ [6585190] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585190/ [6585191] rados/cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585191/ [6585192] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:12:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585192/ [6585193] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585193/ [6585194] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585194/ [6585195] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} tasks/rados_workunit_loadgen_mostlyread} time: 00:28:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585195/ [6585196] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:50:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585196/ [6585197] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585197/ [6585198] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:32:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585198/ [6585199] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:25:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585199/ [6585200] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:22:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585200/ [6585201] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/dashboard} time: 01:13:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585201/ [6585202] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/crash} time: 00:12:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585202/ [6585203] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:20:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585203/ [6585204] rados/rest/{mgr-restful supported-random-distro$/{centos_8.stream}} time: 00:11:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585204/ [6585205] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585205/ [6585206] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/c2c} time: 00:11:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585206/ [6585207] rados/upgrade/parallel/{0-random-distro$/{rhel_8.4_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:31:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585207/ [6585208] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:26:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585208/ [6585209] rados/cephadm/workunits/{agent/off mon_election/classic task/test_adoption} time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585209/ [6585210] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585210/ [6585211] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585211/ [6585212] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:07:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585212/ [6585213] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585213/ [6585214] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585214/ [6585215] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585215/ [6585216] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585216/ [6585217] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585217/ [6585218] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585218/ [6585219] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects} time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585219/ [6585220] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli} time: 00:27:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585220/ [6585221] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:12:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585221/ [6585222] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 01:28:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585222/ [6585223] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585223/ [6585224] rados/multimon/{clusters/9 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:06:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585224/ [6585225] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:37:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585225/ [6585226] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:24:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585226/ [6585227] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:47:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585227/ [6585228] rados/cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585228/ [6585229] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585229/ [6585230] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:20:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585230/ [6585231] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} tasks/readwrite} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585231/ [6585232] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:09:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585232/ [6585233] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8.stream} tasks/failover} time: 00:13:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585233/ [6585234] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:11:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585234/ [6585235] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-overwrites} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585235/ [6585236] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm} time: 00:15:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585236/ [6585237] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585237/ [6585238] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585238/ [6585239] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585239/ [6585240] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585240/ [6585241] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8.stream}} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585241/ [6585242] rados/cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:49:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585242/ [6585243] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585243/ [6585244] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream}} time: 00:08:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585244/ [6585245] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585245/ [6585246] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:34:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585246/ [6585247] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585247/ [6585248] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:12:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585248/ [6585249] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585249/ [6585250] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/crush} time: 00:20:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585250/ [6585251] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:10:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585251/ [6585252] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:45:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585252/ [6585253] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:31:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585253/ [6585254] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585254/ [6585255] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind} time: 00:51:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585255/ [6585256] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:48:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585256/ [6585257] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585257/ [6585258] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:37:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585258/ [6585259] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585259/ [6585260] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585260/ [6585261] rados/cephadm/workunits/{agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585261/ [6585262] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:20:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585262/ [6585263] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{filestore-xfs} supported-random-distro$/{centos_8.stream} tasks/insights} time: 00:13:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585263/ [6585264] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585264/ [6585265] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585265/ [6585266] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:32:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585266/ [6585267] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:28:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585267/ [6585268] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:28:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585268/ [6585269] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585269/ [6585270] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585270/ [6585271] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585271/ [6585272] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585272/ [6585273] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585273/ [6585274] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:18:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585274/ [6585275] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:07:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585275/ [6585276] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:35:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585276/ [6585277] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585277/ [6585278] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585278/ [6585279] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585279/ [6585280] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:41:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585280/ [6585281] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:31:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585281/ [6585282] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585282/ [6585283] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585283/ [6585284] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 01:44:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585284/ [6585285] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585285/ [6585286] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585286/ [6585287] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:18:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585287/ [6585288] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/flannel rook/master} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585288/ [6585289] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:33:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585289/ [6585290] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585290/ [6585291] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_8.stream} tasks/module_selftest} time: 00:31:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585291/ [6585292] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:15:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585292/ [6585293] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_nfs} time: 00:27:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585293/ [6585294] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/lockdep} time: 00:18:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585294/ [6585295] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585295/ [6585296] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 01:27:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585296/ [6585297] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585297/ [6585298] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:28:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585298/ [6585299] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585299/ [6585300] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585300/ [6585301] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585301/ [6585302] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585302/ [6585303] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:30:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585303/ [6585304] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:42:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585304/ [6585305] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:37:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585305/ [6585306] rados/multimon/{clusters/3 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream} tasks/mon_clock_no_skews} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585306/ [6585308] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585308/ [6585309] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585309/ [6585310] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:20:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585310/ [6585311] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/snaps-few-objects} time: 00:36:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585311/ [6585312] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585312/ [6585313] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585313/ [6585314] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585314/ [6585315] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:51:58 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585315/ [6585316] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:41:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585316/ [6585317] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585317/ [6585318] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585318/ [6585319] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:53:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585319/ [6585320] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585320/ [6585321] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{rhel_8} tasks/progress} time: 00:35:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585321/ [6585322] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585322/ [6585323] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585323/ [6585324] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585324/ [6585325] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:29:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585325/ [6585326] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585326/ [6585327] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} tasks/libcephsqlite} time: 00:17:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585327/ [6585328] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:18:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585328/ [6585329] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585329/ [6585330] rados/cephadm/workunits/{agent/off mon_election/classic task/test_orch_cli} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585330/ [6585331] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:50:32 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585331/ [6585332] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585332/ [6585333] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:22:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585333/ [6585334] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585334/ [6585335] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/valgrind} time: 00:40:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585335/ [6585336] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:22:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585336/ [6585337] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585337/ [6585338] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585338/ [6585339] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585339/ [6585340] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:14:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585340/ [6585341] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585341/ [6585342] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585342/ [6585343] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:26:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585343/ [6585344] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 01:00:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585344/ [6585345] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:44 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585345/ [6585346] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585346/ [6585347] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:08:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585347/ [6585348] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:16:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585348/ [6585349] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} thrashers/one workloads/pool-create-delete} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585349/ [6585350] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:33:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585350/ [6585352] rados/singleton/{all/peer mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585352/ [6585353] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{filestore-xfs} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:10:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585353/ [6585354] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:21:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585354/ [6585355] rados/cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585355/ [6585356] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:13:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585356/ [6585357] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:23:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585357/ [6585358] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585358/ [6585359] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:25:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585359/ [6585360] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:35:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585360/ [6585361] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_adoption} time: 00:07:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585361/ [6585362] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585362/ [6585363] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:19:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585363/ [6585364] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585364/ [6585365] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585365/ [6585366] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585366/ [6585367] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:29:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585367/ [6585368] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585368/ [6585369] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:23:47 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585369/ [6585370] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:33:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585370/ [6585371] rados/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:36:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585371/ [6585372] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585372/ [6585373] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585373/ [6585374] rados/cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585374/ [6585375] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585375/ [6585376] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache} time: 00:15:22 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585376/ [6585377] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585377/ [6585378] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/lockdep} time: 00:26:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585378/ [6585379] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585379/ [6585380] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:07:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585380/ [6585381] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585381/ [6585382] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream}} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585382/ [6585383] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585383/ [6585384] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585384/ [6585385] rados/cephadm/workunits/{agent/off mon_election/classic task/test_cephadm} time: 00:15:07 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585385/ [6585386] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:52:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585386/ [6585387] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:53:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585387/ [6585388] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} tasks/rados_cls_all} time: 00:15:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585388/ [6585389] rados/multimon/{clusters/9 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585389/ [6585390] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/dedup-io-mixed} time: 00:23:02 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585390/ [6585391] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:35:46 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585391/ [6585392] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} thrashers/sync-many workloads/rados_5925} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585392/ [6585393] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585393/ [6585394] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585394/ [6585395] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/host rook/1.7.2} time: 00:16:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585395/ [6585396] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:16:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585396/ [6585397] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585397/ [6585398] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:19 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585398/ [6585399] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585399/ [6585400] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585400/ [6585401] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} time: 00:24:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585401/ [6585402] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585402/ [6585403] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:52 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585403/ [6585404] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585404/ [6585405] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:44:31 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585405/ [6585406] rados/singleton/{all/random-eio mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:18:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585406/ [6585407] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:33:04 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585407/ [6585408] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:32:59 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585408/ [6585409] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:32:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585409/ [6585410] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:21:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585410/ [6585411] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:14:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585411/ [6585412] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{centos_8} tasks/crash} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585412/ [6585413] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:19:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585413/ [6585414] rados/upgrade/parallel/{0-random-distro$/{rhel_8.4_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:31:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585414/ [6585415] rados/cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:36:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585415/ [6585416] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:41 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585416/ [6585417] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:04:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585417/ [6585418] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:40:53 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585418/ [6585419] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:15:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585419/ [6585420] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585420/ [6585421] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585421/ [6585422] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/rados_cls_all validater/valgrind} time: 02:02:20 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585422/ [6585423] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} tasks/rados_python} time: 00:14:13 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585423/ [6585424] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585424/ [6585425] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585425/ [6585426] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:56 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585426/ [6585427] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:11 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585427/ [6585428] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:11:51 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585428/ [6585429] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:17:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585429/ [6585430] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr agent/off orchestrator_cli} time: 00:26:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585430/ [6585431] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:31:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585431/ [6585432] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:25:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585432/ [6585433] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:21:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585433/ [6585434] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585434/ [6585435] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_api_tests} time: 00:26:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585435/ [6585436] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585436/ [6585437] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:10:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585437/ [6585438] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:11:16 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585438/ [6585439] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:36:00 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585439/ [6585440] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:10:35 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585440/ [6585441] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585441/ [6585442] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{rhel_8} tasks/failover} time: 00:25:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585442/ [6585443] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:49:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585443/ [6585444] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:08:10 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585444/ [6585445] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585445/ [6585446] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585446/ [6585447] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585447/ [6585448] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:06 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585448/ [6585449] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585449/ [6585450] rados/cephadm/workunits/{agent/off mon_election/classic task/test_nfs} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585450/ [6585451] rados/singleton/{all/test-crash mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream}} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585451/ [6585452] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:21:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585452/ [6585453] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:34 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585453/ [6585454] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} tasks/rados_stress_watch} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585454/ [6585455] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585455/ [6585456] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/radosbench} time: 01:23:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585456/ [6585457] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585457/ [6585458] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585458/ [6585459] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:05 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585459/ [6585460] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/lockdep} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585460/ [6585461] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:31:39 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585461/ [6585462] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:17:40 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585462/ [6585463] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:48:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585463/ [6585464] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:27:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585464/ [6585465] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585465/ [6585466] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8.stream}} time: 00:23:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585466/ [6585467] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:54 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585467/ [6585468] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:33 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585468/ [6585469] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:25:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585469/ [6585470] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{centos_8.stream} tasks/insights} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585470/ [6585471] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:24 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585471/ [6585472] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:56:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585472/ [6585473] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585473/ [6585474] rados/multimon/{clusters/3 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:06:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585474/ [6585475] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:21 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585475/ [6585476] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:47:38 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585476/ [6585477] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:08 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585477/ [6585479] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:28 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585479/ [6585480] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/osd-backfill} time: 04:18:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585480/ [6585481] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:26:57 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585481/ [6585483] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:20:36 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585483/ [6585484] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:17:12 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585484/ [6585485] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:07:29 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585485/ [6585486] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585486/ [6585487] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:28:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585487/ [6585488] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585488/ [6585489] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585489/ [6585490] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:49:03 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585490/ [6585491] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:20:45 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585491/ [6585492] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:48 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585492/ [6585493] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:38:01 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585493/ [6585494] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:31:27 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585494/ [6585495] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:24:14 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585495/ [6585496] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:43 info: http://pulpito.ceph.com/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585496/