2023-06-09T22:00:21.488 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-09T22:00:21.489 INFO:teuthology.results:subset: '111/120000' 2023-06-09T22:00:21.489 INFO:teuthology.results:seed: '6439' 2023-06-10T04:51:25.020 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-10T04:51:25.081 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-10T12:10:05.370 INFO:teuthology.results:Tests finished! gathering results... 2023-06-10T12:10:29.609 INFO:scrape:Found 311 jobs 2023-06-10T12:10:29.611 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300327/teuthology.log 2023-06-10T12:10:31.353 INFO:scrape:Found 8 distinct failure reasons 2023-06-10T12:10:31.354 INFO:scrape:Dead: None 2023-06-10T12:10:31.354 INFO:scrape:5 jobs: ['7300327', '7300484', '7300345', '7300380', '7300346'] 2023-06-10T12:10:31.354 INFO:scrape: 2023-06-10T12:10:31.354 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi099 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-06-10T12:10:31.355 INFO:scrape:2 jobs: ['7300526', '7300432'] 2023-06-10T12:10:31.355 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'test_envlibrados_for_rocksdb}'] 2023-06-10T12:10:31.355 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{ubuntu_20.04}}', 'supported-random-distro$/{ubuntu_latest}}', 'test_envlibrados_for_rocksdb}'] 2023-06-10T12:10:31.356 INFO:scrape: 2023-06-10T12:10:31.356 INFO:scrape:Failure: Command failed on smithi061 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 2023-06-10T12:10:31.356 INFO:scrape:4 jobs: ['7300309', '7300418', '7300161', '7300576'] 2023-06-10T12:10:31.357 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04'] 2023-06-10T12:10:31.357 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] 2023-06-10T12:10:31.357 INFO:scrape: 2023-06-10T12:10:31.358 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-06-10T12:10:31.358 INFO:scrape:2 jobs: ['7300307', '7300572'] 2023-06-10T12:10:31.358 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-06-10T12:10:31.358 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-lz4}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2023-06-10T12:10:31.359 INFO:scrape: 2023-06-10T12:10:31.359 INFO:scrape:Failure: Command failed on smithi102 with status 1: 'sudo yum install -y kernel' 2023-06-10T12:10:31.359 INFO:scrape:2 jobs: ['7300365', '7300328'] 2023-06-10T12:10:31.360 INFO:scrape:suites intersection: [] 2023-06-10T12:10:31.360 INFO:scrape:suites union: ['0-nvme-loop', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-recovery}', '3-scrub-overrides/{default}', 'agent/on', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'start}', 'supported-random-distro$/{centos_8}', 'thrashers/careful', 'thrashosds-health', 'workloads/snaps-few-objects-balanced}'] 2023-06-10T12:10:31.360 INFO:scrape: 2023-06-10T12:10:31.361 INFO:scrape:Dead: b'2023-06-10T12:06:58.629 DEBUG:teuthology.exit:Finished running handlers' 2023-06-10T12:10:31.361 INFO:scrape:1 jobs: ['7300215'] 2023-06-10T12:10:31.361 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-async-partial-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2023-06-10T12:10:31.361 INFO:scrape: 2023-06-10T12:10:31.362 INFO:scrape:Failure: failed to install new kernel version within timeout 2023-06-10T12:10:31.362 INFO:scrape:1 jobs: ['7300362'] 2023-06-10T12:10:31.362 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/rbd_cls}'] 2023-06-10T12:10:31.363 INFO:scrape: 2023-06-10T12:10:31.363 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2023-06-10T12:10:31.363 INFO:scrape:1 jobs: ['7300326'] 2023-06-10T12:10:31.363 INFO:scrape:suites: ['mon_election/classic', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{ubuntu_latest}}'] 2023-06-10T12:10:31.364 INFO:scrape: 2023-06-10T12:10:31.376 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/ failed: 11 dead: 7 running: 0 waiting: 0 queued: 0 passed: 293 Fail ================================================================= [7300161] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:05 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300161/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300161/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5294f26883544f75938f3cf96b87a20f Command failed on smithi155 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7300307] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} tasks/e2e} ----------------------------------------------------------------- time: 00:20:33 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300307/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300307/ sentry: https://sentry.ceph.com/organizations/ceph/?query=739278f53a6a4915a3a3c1f835ff3f95 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7300309] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300309/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300309/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e458bd2c16c740db89019efa63637431 Command failed on smithi061 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7300326] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:19 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300326/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300326/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5b5c327f3a27491db153000447426211 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7300362] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300362/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300362/ sentry: https://sentry.ceph.com/organizations/ceph/?query=af1944751623403a8a011aaf88e677ba failed to install new kernel version within timeout [7300365] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300365/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300365/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4625bfbf1335419090380bb4f73511e0 Command failed on smithi102 with status 1: 'sudo yum install -y kernel' [7300418] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300418/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300418/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7affc111fafc4ffcbbbb1f886ac7d352 Command failed on smithi064 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7300432] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:12:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300432/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300432/ sentry: https://sentry.ceph.com/organizations/ceph/?query=769347af16f74a61b675c012cb87f635 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi132 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7300526] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:10:13 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300526/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300526/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ca7c7ebe786f450592cb60757ef394cd Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi099 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7300572] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:21:06 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300572/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300572/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b6e69154409e46c3be08ea03bd478313 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5937c68aed42cde57f938ad8f8bb264f13176009 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7300576] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:10 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300576/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300576/ sentry: https://sentry.ceph.com/organizations/ceph/?query=085a7cdbb6e64e48ac21f116365a1f3e Command failed on smithi040 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" Dead ================================================================= [7300215] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300215/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300215/ hit max job timeout [7300327] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/module_selftest} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300327/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300327/ Error reimaging machines: Expected smithi102's OS to be ubuntu 20.04 but found centos 8 [7300328] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300328/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300328/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40f1c0a8b34e45d6ab9fb2e7416b9cbe SSH connection to smithi102 was lost: 'sudo yum install -y kernel' [7300345] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300345/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300345/ Error reimaging machines: Expected smithi102's OS to be rhel 8.6 but found ubuntu 20.04 [7300346] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300346/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300346/ Error reimaging machines: [7300380] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300380/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300380/ Error reimaging machines: Expected smithi102's OS to be centos 8 but found ubuntu 20.04 [7300484] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300484/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300484/ Error reimaging machines: Expected smithi163's OS to be centos 8 but found rhel 8.6 Pass ================================================================= [7300046] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:48:38 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300046/ [7300048] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:33:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300048/ [7300050] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300050/ [7300052] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300052/ [7300054] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:36:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300054/ [7300056] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:09:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300056/ [7300058] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:18:24 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300058/ [7300060] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:41 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300060/ [7300062] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300062/ [7300064] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300064/ [7300066] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:24:07 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300066/ [7300068] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{ubuntu_20.04}} time: 00:09:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300068/ [7300070] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:28:21 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300070/ [7300072] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:26:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300072/ [7300074] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300074/ [7300076] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:19:31 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300076/ [7300077] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300077/ [7300079] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300079/ [7300081] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300081/ [7300083] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:20:58 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300083/ [7300085] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:18 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300085/ [7300087] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:21:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300087/ [7300089] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:06:38 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300089/ [7300091] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300091/ [7300093] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:37:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300093/ [7300095] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:25:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300095/ [7300097] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub} time: 02:49:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300097/ [7300099] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:11:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300099/ [7300101] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300101/ [7300103] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:14:49 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300103/ [7300105] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:24:41 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300105/ [7300107] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300107/ [7300109] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300109/ [7300111] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/many workloads/rados_mon_workunits} time: 00:32:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300111/ [7300113] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:35:31 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300113/ [7300115] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300115/ [7300117] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:09:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300117/ [7300119] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:28:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300119/ [7300121] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:31:21 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300121/ [7300123] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300123/ [7300125] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300125/ [7300127] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/rados_api_tests} time: 00:29:21 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300127/ [7300129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:34:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300129/ [7300131] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300131/ [7300133] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:26:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300133/ [7300135] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:34:48 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300135/ [7300137] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300137/ [7300139] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300139/ [7300141] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:34:56 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300141/ [7300143] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:01 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300143/ [7300145] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300145/ [7300147] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300147/ [7300149] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:27:26 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300149/ [7300151] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:08:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300151/ [7300153] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/dashboard} time: 01:11:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300153/ [7300155] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_20.04} tasks/crash} time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300155/ [7300157] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:41:26 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300157/ [7300159] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_20.04}} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300159/ [7300163] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:30:30 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300163/ [7300165] rados/standalone/{supported-random-distro$/{centos_8} workloads/c2c} time: 00:16:24 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300165/ [7300167] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:29:51 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300167/ [7300169] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:22:13 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300169/ [7300171] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:15:59 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300171/ [7300173] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:28:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300173/ [7300175] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300175/ [7300177] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:17:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300177/ [7300179] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:35:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300179/ [7300181] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:27 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300181/ [7300183] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300183/ [7300185] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300185/ [7300187] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/one workloads/snaps-few-objects} time: 00:40:19 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300187/ [7300189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:20:56 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300189/ [7300191] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:17:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300191/ [7300193] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300193/ [7300195] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300195/ [7300197] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/radosbench} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300197/ [7300199] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:28:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300199/ [7300201] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:16:24 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300201/ [7300203] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:29 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300203/ [7300205] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300205/ [7300207] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:21:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300207/ [7300209] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300209/ [7300211] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300211/ [7300213] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:16:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300213/ [7300217] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_20.04} tasks/failover} time: 00:13:38 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300217/ [7300219] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300219/ [7300221] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:30:36 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300221/ [7300223] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300223/ [7300225] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300225/ [7300226] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300226/ [7300228] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:34 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300228/ [7300230] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300230/ [7300232] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:37:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300232/ [7300234] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:49:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300234/ [7300236] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:20:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300236/ [7300238] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300238/ [7300240] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:22:19 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300240/ [7300243] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:24:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300243/ [7300245] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300245/ [7300247] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:15:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300247/ [7300249] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300249/ [7300251] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300251/ [7300253] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} time: 00:23:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300253/ [7300255] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:18:49 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300255/ [7300256] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300256/ [7300258] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300258/ [7300260] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:15:27 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300260/ [7300262] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:18:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300262/ [7300264] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:11:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300264/ [7300266] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300266/ [7300268] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_20.04} tasks/insights} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300268/ [7300270] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:21:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300270/ [7300272] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:18:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300272/ [7300274] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300274/ [7300276] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:36:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300276/ [7300278] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300278/ [7300280] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:25:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300280/ [7300282] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:26:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300282/ [7300285] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:36:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300285/ [7300286] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300286/ [7300287] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:19:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300287/ [7300288] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300288/ [7300289] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:27 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300289/ [7300290] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:27:23 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300290/ [7300291] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300291/ [7300293] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:21:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300293/ [7300295] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:30:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300295/ [7300297] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:01 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300297/ [7300299] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300299/ [7300301] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:11:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300301/ [7300303] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:30 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300303/ [7300305] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300305/ [7300311] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300311/ [7300313] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:35:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300313/ [7300315] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:31:30 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300315/ [7300317] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:24:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300317/ [7300319] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:08:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300319/ [7300321] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:12:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300321/ [7300323] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:17:15 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300323/ [7300325] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300325/ [7300329] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:22:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300329/ [7300330] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:16:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300330/ [7300331] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:17:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300331/ [7300332] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300332/ [7300333] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:33 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300333/ [7300334] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:55:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300334/ [7300335] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:15 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300335/ [7300336] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300336/ [7300337] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:40:10 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300337/ [7300338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300338/ [7300339] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300339/ [7300340] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300340/ [7300341] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300341/ [7300342] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300342/ [7300343] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:25:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300343/ [7300344] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:35:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300344/ [7300347] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:30:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300347/ [7300348] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:13 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300348/ [7300349] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:33:06 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300349/ [7300350] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300350/ [7300351] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:24:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300351/ [7300352] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} time: 00:34:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300352/ [7300353] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:41:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300353/ [7300354] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300354/ [7300355] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:20:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300355/ [7300356] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/per_module_finisher_stats} time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300356/ [7300357] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:38 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300357/ [7300358] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300358/ [7300359] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:25:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300359/ [7300360] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:19:21 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300360/ [7300361] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300361/ [7300363] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} time: 01:11:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300363/ [7300364] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300364/ [7300366] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300366/ [7300367] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300367/ [7300368] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300368/ [7300369] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:26:01 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300369/ [7300370] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} time: 00:24:05 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300370/ [7300371] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:38:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300371/ [7300372] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:19:51 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300372/ [7300374] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:25:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300374/ [7300376] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:36:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300376/ [7300378] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:33:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300378/ [7300382] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300382/ [7300384] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:24 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300384/ [7300386] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300386/ [7300388] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300388/ [7300390] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:15:23 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300390/ [7300392] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:32:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300392/ [7300394] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:26:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300394/ [7300396] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:37:59 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300396/ [7300398] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:47 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300398/ [7300400] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:01 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300400/ [7300402] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:18 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300402/ [7300404] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:00:23 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300404/ [7300406] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:27:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300406/ [7300408] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300408/ [7300410] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300410/ [7300412] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:23:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300412/ [7300414] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300414/ [7300416] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} time: 01:16:15 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300416/ [7300420] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:19:15 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300420/ [7300422] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon-stretch} time: 00:20:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300422/ [7300424] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:28:34 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300424/ [7300426] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:16:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300426/ [7300428] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:28:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300428/ [7300430] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300430/ [7300434] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300434/ [7300436] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:27:45 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300436/ [7300438] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:56 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300438/ [7300440] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:52 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300440/ [7300442] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:41:36 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300442/ [7300444] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:20:41 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300444/ [7300446] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:54:48 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300446/ [7300448] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:23:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300448/ [7300450] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300450/ [7300452] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300452/ [7300454] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300454/ [7300456] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:12:56 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300456/ [7300458] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300458/ [7300460] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:53 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300460/ [7300462] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:23:50 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300462/ [7300464] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{rhel_8}} time: 02:16:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300464/ [7300466] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:22:10 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300466/ [7300468] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300468/ [7300470] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:28 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300470/ [7300472] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300472/ [7300474] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:18:18 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300474/ [7300476] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits} time: 00:30:04 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300476/ [7300478] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:25:46 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300478/ [7300480] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:13:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300480/ [7300482] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300482/ [7300486] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300486/ [7300488] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:08:41 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300488/ [7300490] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300490/ [7300492] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:28:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300492/ [7300494] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300494/ [7300496] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300496/ [7300498] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:36:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300498/ [7300500] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:25:14 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300500/ [7300502] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300502/ [7300504] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300504/ [7300506] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300506/ [7300508] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/repair_test} time: 00:17:20 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300508/ [7300510] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:40:13 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300510/ [7300512] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300512/ [7300514] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_20.04} tasks/workunits} time: 00:10:02 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300514/ [7300516] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:16:30 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300516/ [7300518] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:22:23 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300518/ [7300520] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:07 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300520/ [7300522] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:24:51 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300522/ [7300524] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300524/ [7300528] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} time: 00:22:34 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300528/ [7300530] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:22:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300530/ [7300532] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300532/ [7300534] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:24:36 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300534/ [7300536] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:11:45 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300536/ [7300538] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:47 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300538/ [7300540] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 02:52:08 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300540/ [7300541] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:36:11 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300541/ [7300543] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:18:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300543/ [7300545] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300545/ [7300547] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_20.04}} time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300547/ [7300549] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300549/ [7300551] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:27:24 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300551/ [7300553] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:26:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300553/ [7300555] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:31 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300555/ [7300558] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300558/ [7300559] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300559/ [7300561] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:06:57 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300561/ [7300563] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:22:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300563/ [7300566] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:23:45 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300566/ [7300568] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:37 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300568/ [7300570] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:28:46 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300570/ [7300574] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300574/ [7300577] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:35 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300577/ [7300579] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:18:42 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300579/ [7300581] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:28:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300581/ [7300583] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:16 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300583/ [7300585] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:17:17 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300585/ [7300587] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300587/ [7300590] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:43 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300590/ [7300592] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:42:25 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300592/ [7300594] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300594/ [7300596] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:55:03 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300596/ [7300598] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300598/ [7300600] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:22 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300600/ [7300602] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:56 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300602/ [7300604] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 02:57:55 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300604/ [7300606] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300606/ [7300608] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:13:12 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300608/ [7300610] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/redirect} time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300610/ [7300612] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:39 info: http://pulpito.ceph.com/yuriw-2023-06-09_20:58:19-rados-reef-distro-default-smithi/7300612/