2023-05-20T11:20:01.161 INFO:root:teuthology version: 0.0.1.dev100+g01abd1e 2023-05-20T11:20:01.163 INFO:teuthology.results:subset: '111/120000' 2023-05-20T11:20:01.163 INFO:teuthology.results:seed: '1421' 2023-05-20T17:37:19.886 INFO:root:teuthology version: 0.0.1.dev100+g01abd1e 2023-05-20T17:37:19.910 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-21T03:31:18.908 INFO:teuthology.results:Tests finished! gathering results... 2023-05-21T03:31:45.337 INFO:scrape:Found 309 jobs 2023-05-21T03:31:48.887 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279917/teuthology.log 2023-05-21T03:31:50.102 INFO:scrape:Found 10 distinct failure reasons 2023-05-21T03:31:50.103 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-05-21T03:31:50.103 INFO:scrape:2 jobs: ['7279966', '7280230'] 2023-05-21T03:31:50.103 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-05-21T03:31:50.104 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-bitmap}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2023-05-21T03:31:50.104 INFO:scrape: 2023-05-21T03:31:50.104 INFO:scrape:Dead: b'2023-05-21T03:27:33.347 DEBUG:teuthology.exit:Finished running handlers' 2023-05-21T03:31:50.105 INFO:scrape:1 jobs: ['7280055'] 2023-05-21T03:31:50.105 INFO:scrape:suites: ['debug/mgr', 'mgr_ttl_cache/enable', 'mon_election/connectivity', 'rados/mgr/{clusters/{2-node-mgr}', 'random-objectstore$/{bluestore-hybrid}', 'supported-random-distro$/{rhel_8}', 'tasks/progress}'] 2023-05-21T03:31:50.105 INFO:scrape: 2023-05-21T03:31:50.105 INFO:scrape:Failure: Command failed on smithi098 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' 2023-05-21T03:31:50.106 INFO:scrape:1 jobs: ['7279833'] 2023-05-21T03:31:50.106 INFO:scrape:suites: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', 'cluster/1-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}'] 2023-05-21T03:31:50.106 INFO:scrape: 2023-05-21T03:31:50.107 INFO:scrape:Failure: Config file not found: "/home/teuthworker/src/github.com_ceph_ceph-c_968e70df8df0613a56f366bea5f3f7e24dbabf8a/qa/tasks/cephadm.conf". 2023-05-21T03:31:50.107 INFO:scrape:1 jobs: ['7279798'] 2023-05-21T03:31:50.107 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_20.04'] 2023-05-21T03:31:50.107 INFO:scrape: 2023-05-21T03:31:50.108 INFO:scrape:Failure: Command failed on smithi044 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 2023-05-21T03:31:50.108 INFO:scrape:3 jobs: ['7280078', '7280234', '7279967'] 2023-05-21T03:31:50.108 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04'] 2023-05-21T03:31:50.109 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] 2023-05-21T03:31:50.109 INFO:scrape: 2023-05-21T03:31:50.109 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rbd.sh) on smithi035 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' 2023-05-21T03:31:50.110 INFO:scrape:2 jobs: ['7279839', '7280082'] 2023-05-21T03:31:50.110 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2023-05-21T03:31:50.110 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2023-05-21T03:31:50.111 INFO:scrape: 2023-05-21T03:31:50.111 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2023-05-21T03:31:50.111 INFO:scrape:2 jobs: ['7279976', '7280012'] 2023-05-21T03:31:50.112 INFO:scrape:suites intersection: ['mon_election/classic', 'rados'] 2023-05-21T03:31:50.112 INFO:scrape:suites union: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/rbd_cls}'] 2023-05-21T03:31:50.112 INFO:scrape: 2023-05-21T03:31:50.113 INFO:scrape:Failure: No module named 'tasks.ceph' 2023-05-21T03:31:50.113 INFO:scrape:3 jobs: ['7279797', '7279801', '7279792'] 2023-05-21T03:31:50.113 INFO:scrape:suites intersection: [] 2023-05-21T03:31:50.114 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'agent/on', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'rados/singleton-nomsgr/{all/pool-access', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'task/test_rgw_multisite}', 'thrashers/careful', 'thrashosds-health', 'workloads/dedup-io-snaps}'] 2023-05-21T03:31:50.114 INFO:scrape: 2023-05-21T03:31:50.114 INFO:scrape:Dead: None 2023-05-21T03:31:50.115 INFO:scrape:1 jobs: ['7279917'] 2023-05-21T03:31:50.115 INFO:scrape: 2023-05-21T03:31:50.115 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-05-21T03:31:50.115 INFO:scrape:2 jobs: ['7280186', '7280090'] 2023-05-21T03:31:50.116 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'supported-random-distro$/{rhel_8}}', 'test_envlibrados_for_rocksdb}'] 2023-05-21T03:31:50.116 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{rhel_8}}', 'test_envlibrados_for_rocksdb}'] 2023-05-21T03:31:50.116 INFO:scrape: 2023-05-21T03:31:50.128 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/ failed: 16 dead: 2 running: 0 waiting: 0 queued: 0 passed: 291 Fail ================================================================= [7279792] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_rgw_multisite} ----------------------------------------------------------------- time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279792/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279792/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50d6da99e7aa4043928242928d21228b No module named 'tasks.cephadm' [7279797] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279797/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279797/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c62fdfd39281458bab9270ea047bbe30 No module named 'tasks.ceph' [7279798] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:06:21 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279798/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279798/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9364a020aef847ed887f64dd01a73cb2 Config file not found: "/home/teuthworker/src/github.com_ceph_ceph- c_968e70df8df0613a56f366bea5f3f7e24dbabf8a/qa/tasks/cephadm.conf". [7279801] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-snaps} ----------------------------------------------------------------- time: 00:16:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279801/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279801/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ef807a493b2648b2bc3c8cf606b2be31 No module named 'tasks' [7279833] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:27 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279833/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279833/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e72f0ca06b64193a74f35e70715c02b Command failed on smithi098 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7279839] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:47:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279839/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279839/ sentry: https://sentry.ceph.com/organizations/ceph/?query=afa06d5540034ba292d814be4171aed6 Command failed (workunit test cls/test_cls_rbd.sh) on smithi035 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7279966] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:25:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279966/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279966/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02c05db02fc643a8bd8a40e68c9d8fbc Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7279967] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:02 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279967/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279967/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4cdd7a58b6e246e2b78455804361f935 Command failed on smithi044 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7279976] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279976/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279976/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ca6a8abeb78c49979fa0945de1fb2460 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7280012] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280012/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280012/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9078615410d542e9a2d4ddeec03d688b Command failed on smithi186 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-Te stClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" [7280078] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280078/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280078/ sentry: https://sentry.ceph.com/organizations/ceph/?query=22fa7c7cd2b3495cb1c2c99d16b97a4f Command failed on smithi044 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7280082] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:43:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280082/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280082/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1c5c3959ff3141799f066d558a7fc567 Command failed (workunit test cls/test_cls_rbd.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7280090] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280090/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280090/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ef5788ac991743c78bb66205bb6435c5 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7280186] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:19:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280186/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280186/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ae27adafd5b4bf1bc2a8f46852109de Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7280230] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/e2e} ----------------------------------------------------------------- time: 00:25:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280230/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280230/ sentry: https://sentry.ceph.com/organizations/ceph/?query=925721e06eea4f1492e7107d9eacfc71 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=968e70df8df0613a56f366bea5f3f7e24dbabf8a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7280234] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:12 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280234/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280234/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d975d60f41ea450dad93b2777456cd95 Command failed on smithi158 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" Dead ================================================================= [7279917] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279917/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279917/ Error reimaging machines: 'NoneType' object has no attribute '_fields' [7280055] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{rhel_8} tasks/progress} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280055/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280055/ hit max job timeout Pass ================================================================= [7279786] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:20:15 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279786/ [7279787] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:31:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279787/ [7279788] rados/standalone/{supported-random-distro$/{rhel_8} workloads/scrub} time: 02:44:18 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279788/ [7279789] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:09:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279789/ [7279790] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279790/ [7279791] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279791/ [7279793] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:18:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279793/ [7279794] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/workunits} time: 00:18:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279794/ [7279795] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:12:02 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279795/ [7279796] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:26:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279796/ [7279799] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279799/ [7279800] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/many workloads/rados_mon_workunits} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279800/ [7279802] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279802/ [7279803] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279803/ [7279804] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:30:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279804/ [7279805] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:33:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279805/ [7279806] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:06 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279806/ [7279807] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279807/ [7279808] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:30:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279808/ [7279809] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:18 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279809/ [7279810] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:25:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279810/ [7279811] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:33:02 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279811/ [7279812] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279812/ [7279813] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:26:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279813/ [7279814] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:34:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279814/ [7279816] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279816/ [7279818] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:21 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279818/ [7279820] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279820/ [7279822] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:38:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279822/ [7279824] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:10:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279824/ [7279826] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} tasks/dashboard} time: 01:05:45 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279826/ [7279827] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/crash} time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279827/ [7279829] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_20.04}} time: 00:39:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279829/ [7279831] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:22:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279831/ [7279835] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:20:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279835/ [7279837] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/c2c} time: 00:13:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279837/ [7279841] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:25:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279841/ [7279843] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:16:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279843/ [7279845] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm} time: 00:18:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279845/ [7279847] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:25:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279847/ [7279849] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279849/ [7279851] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279851/ [7279853] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:47:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279853/ [7279855] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279855/ [7279857] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279857/ [7279859] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/snaps-few-objects} time: 00:34:48 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279859/ [7279861] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:22:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279861/ [7279863] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:19:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279863/ [7279865] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:17:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279865/ [7279867] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279867/ [7279869] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench} time: 00:38:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279869/ [7279871] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279871/ [7279873] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279873/ [7279875] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm_repos} time: 00:10:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279875/ [7279877] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279877/ [7279879] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:21:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279879/ [7279881] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279881/ [7279883] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279883/ [7279885] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:25:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279885/ [7279887] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279887/ [7279889] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/failover} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279889/ [7279891] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279891/ [7279893] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279893/ [7279895] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279895/ [7279897] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279897/ [7279899] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279899/ [7279901] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279901/ [7279902] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:14:40 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279902/ [7279903] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:34:27 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279903/ [7279904] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:48:18 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279904/ [7279905] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:22:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279905/ [7279906] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279906/ [7279907] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:23:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279907/ [7279909] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:14:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279909/ [7279911] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279911/ [7279913] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:21:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279913/ [7279915] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_20.04}} time: 00:08:21 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279915/ [7279919] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279919/ [7279920] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/sync-many workloads/pool-create-delete} time: 00:18:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279920/ [7279922] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279922/ [7279924] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_nfs} time: 00:30:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279924/ [7279926] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279926/ [7279928] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:18:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279928/ [7279930] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279930/ [7279931] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:17:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279931/ [7279932] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:29 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279932/ [7279933] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279933/ [7279934] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} tasks/rados_stress_watch} time: 00:19:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279934/ [7279935] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279935/ [7279936] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:31:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279936/ [7279938] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:58 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279938/ [7279940] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:27:07 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279940/ [7279942] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279942/ [7279944] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279944/ [7279946] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279946/ [7279948] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:37:56 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279948/ [7279950] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:34:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279950/ [7279952] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279952/ [7279954] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:11 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279954/ [7279956] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 01:23:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279956/ [7279958] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:20:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279958/ [7279960] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:20:20 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279960/ [7279961] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:28:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279961/ [7279962] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279962/ [7279963] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279963/ [7279964] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:10:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279964/ [7279965] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:18 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279965/ [7279968] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279968/ [7279969] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:47:15 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279969/ [7279970] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:21:22 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279970/ [7279971] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279971/ [7279972] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli_mon} time: 00:37:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279972/ [7279973] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/rados_striper} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279973/ [7279974] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:22:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279974/ [7279975] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:08:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279975/ [7279977] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/module_selftest} time: 00:26:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279977/ [7279978] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:35:45 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279978/ [7279979] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_20.04}} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279979/ [7279980] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:16:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279980/ [7279981] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279981/ [7279982] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279982/ [7279983] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:18:30 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279983/ [7279984] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279984/ [7279985] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:11:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279985/ [7279986] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279986/ [7279987] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:33:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279987/ [7279988] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279988/ [7279989] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279989/ [7279990] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:15:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279990/ [7279991] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:27 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279991/ [7279992] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} time: 00:20:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279992/ [7279993] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279993/ [7279994] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279994/ [7279995] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:32:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279995/ [7279996] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279996/ [7279997] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:32:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279997/ [7279998] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:22:48 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279998/ [7279999] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_big} time: 00:31:40 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7279999/ [7280000] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280000/ [7280001] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:26:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280001/ [7280002] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:29:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280002/ [7280003] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:27:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280003/ [7280004] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} time: 00:11:18 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280004/ [7280005] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280005/ [7280006] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/per_module_finisher_stats} time: 00:11:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280006/ [7280007] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:25:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280007/ [7280008] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews} time: 00:08:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280008/ [7280009] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280009/ [7280010] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280010/ [7280011] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280011/ [7280014] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280014/ [7280016] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280016/ [7280018] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:14 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280018/ [7280020] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280020/ [7280022] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:28 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280022/ [7280024] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:25:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280024/ [7280026] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mix} time: 00:21:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280026/ [7280028] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:28:43 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280028/ [7280030] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:21:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280030/ [7280032] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} time: 00:20:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280032/ [7280034] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:47:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280034/ [7280036] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:34:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280036/ [7280038] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 01:05:21 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280038/ [7280041] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:27:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280041/ [7280042] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:18:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280042/ [7280044] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280044/ [7280047] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280047/ [7280049] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280049/ [7280051] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280051/ [7280053] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:35:11 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280053/ [7280057] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280057/ [7280059] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280059/ [7280061] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280061/ [7280063] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:17 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280063/ [7280065] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:58:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280065/ [7280067] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280067/ [7280069] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280069/ [7280070] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280070/ [7280072] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:20:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280072/ [7280074] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:34:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280074/ [7280076] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} time: 01:10:31 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280076/ [7280080] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280080/ [7280084] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:14:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280084/ [7280086] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:32:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280086/ [7280088] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280088/ [7280092] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280092/ [7280094] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:24:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280094/ [7280096] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280096/ [7280098] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:15 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280098/ [7280101] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:35:48 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280101/ [7280102] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:11 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280102/ [7280104] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:58:32 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280104/ [7280106] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:33:40 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280106/ [7280108] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280108/ [7280110] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280110/ [7280113] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} time: 01:21:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280113/ [7280114] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:06:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280114/ [7280116] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280116/ [7280118] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_20.04} tasks/prometheus} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280118/ [7280120] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:37:15 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280120/ [7280122] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_20.04}} time: 02:24:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280122/ [7280124] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:26:48 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280124/ [7280126] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/readwrite} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280126/ [7280128] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:23 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280128/ [7280130] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280130/ [7280132] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280132/ [7280134] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits} time: 00:27:57 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280134/ [7280136] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:48 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280136/ [7280138] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_nfs} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280138/ [7280140] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280140/ [7280142] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:11:04 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280142/ [7280144] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:12 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280144/ [7280146] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:29:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280146/ [7280148] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:12:27 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280148/ [7280150] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280150/ [7280152] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:18:30 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280152/ [7280154] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:21:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280154/ [7280156] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:43 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280156/ [7280158] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280158/ [7280160] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:15 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280160/ [7280162] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:13:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280162/ [7280164] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:17:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280164/ [7280166] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:45 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280166/ [7280168] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280168/ [7280170] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_8} tasks/workunits} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280170/ [7280172] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280172/ [7280174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280174/ [7280176] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:14 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280176/ [7280178] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:03 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280178/ [7280180] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/osd-backfill} time: 02:41:27 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280180/ [7280182] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:21:14 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280182/ [7280184] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:16:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280184/ [7280188] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli_mon} time: 00:32:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280188/ [7280190] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:19:00 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280190/ [7280192] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280192/ [7280194] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:26 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280194/ [7280196] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:30 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280196/ [7280198] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:26:07 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280198/ [7280200] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:09 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280200/ [7280202] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:25:33 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280202/ [7280204] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_20.04}} time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280204/ [7280206] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:14:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280206/ [7280208] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:27:52 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280208/ [7280210] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280210/ [7280212] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280212/ [7280214] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280214/ [7280216] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:59 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280216/ [7280218] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/scrub_test} time: 00:12:06 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280218/ [7280220] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:23:10 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280220/ [7280222] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:16:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280222/ [7280224] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:25:10 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280224/ [7280226] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280226/ [7280228] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:22:24 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280228/ [7280232] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/crash} time: 00:11:39 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280232/ [7280236] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280236/ [7280238] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:54 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280238/ [7280240] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280240/ [7280242] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280242/ [7280244] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280244/ [7280246] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280246/ [7280248] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:48:19 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280248/ [7280250] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280250/ [7280252] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 02:47:40 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280252/ [7280254] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:49:10 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280254/ [7280256] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:20:25 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280256/ [7280258] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:10 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280258/ [7280260] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:04 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280260/ [7280262] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280262/ [7280264] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:17:41 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280264/ [7280266] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280266/ [7280268] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280268/ [7280270] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925} time: 00:12:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280270/ [7280272] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280272/ [7280274] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:09:58 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280274/ [7280275] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:33:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280275/ [7280278] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:14:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280278/ [7280280] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:51 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280280/ [7280282] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:15:29 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280282/ [7280284] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:19:34 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280284/ [7280286] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:14:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280286/ [7280288] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/failover} time: 00:19:53 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280288/ [7280290] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:02 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280290/ [7280292] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{rhel_8}} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280292/ [7280294] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:29:36 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280294/ [7280296] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:26:38 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280296/ [7280298] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:37 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280298/ [7280299] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:19:42 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280299/ [7280300] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} time: 00:22:35 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280300/ [7280301] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:36:05 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280301/ [7280302] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:16 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280302/ [7280303] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:29:55 info: http://pulpito.ceph.com/yuriw-2023-05-19_20:43:10-rados-reef-distro-default-smithi/7280303/