2023-03-29T04:43:56.892 INFO:root:teuthology version: 0.0.1.dev78+ge372916 2023-03-29T04:43:56.892 INFO:teuthology.results:subset: '111/120000' 2023-03-29T04:43:56.893 INFO:teuthology.results:seed: '9038' 2023-03-29T09:32:25.167 INFO:root:teuthology version: 0.0.1.dev78+ge372916 2023-03-29T09:32:25.190 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-03-29T21:12:33.540 INFO:teuthology.results:Tests finished! gathering results... 2023-03-29T21:12:58.400 INFO:scrape:Found 305 jobs 2023-03-29T21:12:59.842 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224099/teuthology.log 2023-03-29T21:13:01.837 INFO:scrape:Found 8 distinct failure reasons 2023-03-29T21:13:01.838 INFO:scrape:Failure: Command failed on smithi120 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' 2023-03-29T21:13:01.838 INFO:scrape:6 jobs: ['7224578', '7224186', '7224728', '7224591', '7224506', '7224387'] 2023-03-29T21:13:01.838 INFO:scrape:suites intersection: [] 2023-03-29T21:13:01.838 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', '1-pg-log-overrides/short_pg_log', '1-rook', '2-recovery-overrides/{more-partial-recovery}', '2-workload/none', '2-workload/radosbench', '3-scrub-overrides/{default}', 'backoff/peering', 'ceph', 'cluster/1-node', 'cluster/3-node', 'clusters/{fixed-2', 'clusters/{openstack', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'k8s/1.21', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v2only', 'net/calico', 'net/flannel', 'net/host', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rook/1.7.2}', 'rook/master}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-pool-snaps-readproxy}', 'workloads/rbd_cls}'] 2023-03-29T21:13:01.838 INFO:scrape: 2023-03-29T21:13:01.838 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2023-03-29T21:13:01.838 INFO:scrape:7 jobs: ['7224055', '7224279', '7224155', '7224482', '7224175', '7224707', '7224545'] 2023-03-29T21:13:01.838 INFO:scrape:suites intersection: ['rados'] 2023-03-29T21:13:01.839 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/9-mons', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'd-thrash/default/{default', 'd-thrash/none', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/monthrash/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_20.04}', 'tasks/rados_api_tests', 'tasks/rados_api_tests}', 'thrashers/force-sync-many', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashosds-health', 'thrashosds-health}', 'validater/lockdep}', 'validater/valgrind}', 'workloads/rados_api_tests}'] 2023-03-29T21:13:01.839 INFO:scrape: 2023-03-29T21:13:01.839 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi131.front.sepia.ceph.com: ['type=AVC msg=audit(1680071200.265:20006): avc: denied { ioctl } for pid=109347 comm="iptables" path="/var/lib/containers/storage/overlay/529ab00fc1791ddcd8abaae2be4af11aaa9e78b591423638fe3d6e5c3f08f60c/merged" dev="overlay" ino=3278960 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2023-03-29T21:13:01.839 INFO:scrape:1 jobs: ['7224198'] 2023-03-29T21:13:01.839 INFO:scrape:suites: ['agent/off', 'mon_election/classic', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_cephadm}'] 2023-03-29T21:13:01.839 INFO:scrape: 2023-03-29T21:13:01.839 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-03-29T21:13:01.839 INFO:scrape:2 jobs: ['7224384', '7224724'] 2023-03-29T21:13:01.839 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-03-29T21:13:01.839 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-hybrid}', 'tasks/e2e}'] 2023-03-29T21:13:01.839 INFO:scrape: 2023-03-29T21:13:01.840 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi121 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-03-29T21:13:01.840 INFO:scrape:2 jobs: ['7224687', '7224605'] 2023-03-29T21:13:01.840 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'test_envlibrados_for_rocksdb}'] 2023-03-29T21:13:01.840 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_20.04}}', 'test_envlibrados_for_rocksdb}'] 2023-03-29T21:13:01.840 INFO:scrape: 2023-03-29T21:13:01.840 INFO:scrape:Dead: None 2023-03-29T21:13:01.840 INFO:scrape:1 jobs: ['7224099'] 2023-03-29T21:13:01.840 INFO:scrape: 2023-03-29T21:13:01.840 INFO:scrape:Failure: Command failed (workunit test misc/mclock-config.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh' 2023-03-29T21:13:01.840 INFO:scrape:1 jobs: ['7224543'] 2023-03-29T21:13:01.840 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/misc}'] 2023-03-29T21:13:01.840 INFO:scrape: 2023-03-29T21:13:01.841 INFO:scrape:Dead: b'2023-03-29T21:09:11.535 DEBUG:teuthology.exit:Finished running handlers' 2023-03-29T21:13:01.841 INFO:scrape:1 jobs: ['7224627'] 2023-03-29T21:13:01.841 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/mon}'] 2023-03-29T21:13:01.841 INFO:scrape: 2023-03-29T21:13:01.856 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/ failed: 19 dead: 2 running: 0 waiting: 0 queued: 0 passed: 284 Fail ================================================================= [7224055] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:29:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224055/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224055/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dbb7c9430ab545fcb404b6fc4210c508 Command failed (workunit test rados/test.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224155] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224155/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224155/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ddec32b3b4c441b9859817be3692701c Command failed (workunit test rados/test.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224175] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:32:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224175/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224175/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bbfb285ded5841139bf1ca168c4fbeb2 Command failed (workunit test rados/test.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224186] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224186/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224186/ sentry: https://sentry.ceph.com/organizations/ceph/?query=89f025cfdb334869b54f486af30036a3 Command failed on smithi135 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7224198] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:18:43 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224198/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224198/ SELinux denials found on ubuntu@smithi131.front.sepia.ceph.com: ['type=AVC msg=audit(1680071200.265:20006): avc: denied { ioctl } for pid=109347 comm="iptables" path="/var/lib/containers/storage/overlay/529ab00fc1791ddcd 8abaae2be4af11aaa9e78b591423638fe3d6e5c3f08f60c/merged" dev="overlay" ino=3278960 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7224279] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:47:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224279/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224279/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b5e2585c53d4dd6a95752c21e836867 Command failed (workunit test rados/test.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224384] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} ----------------------------------------------------------------- time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224384/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224384/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8da9b5e3d64a4cc1bdba38d3e052fdba Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7224387] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224387/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224387/ sentry: https://sentry.ceph.com/organizations/ceph/?query=466980663b2047d5950c812dabb5bfc5 Command failed on smithi062 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7224482] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:30:56 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224482/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224482/ sentry: https://sentry.ceph.com/organizations/ceph/?query=291f76a8e75d4e27badceb32bbe7c0f8 Command failed (workunit test rados/test.sh) on smithi088 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224506] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224506/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224506/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c8a64593e53422e9882bf4cc624af7b Command failed on smithi089 with status 5: 'sudo systemctl stop ceph-208d90ca-ce0c-11ed-9afe-001a4aab830c@mon.b' [7224543] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} ----------------------------------------------------------------- time: 00:15:23 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224543/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224543/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b30acf7d45154ea3bb04b1dd8a0c64b9 Command failed (workunit test misc/mclock-config.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh' [7224545] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 00:31:39 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224545/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224545/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7bc69cafde40480897c1e6b576ddc1f0 Command failed (workunit test rados/test.sh) on smithi175 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224578] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224578/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224578/ sentry: https://sentry.ceph.com/organizations/ceph/?query=660b5005adb04535b1e5dab2c97b1d0e Command failed on smithi120 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' [7224591] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:22 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224591/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224591/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1d6c787d0d774f2dbfb221db107c5cb8 Command failed on smithi143 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7224605] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224605/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224605/ sentry: https://sentry.ceph.com/organizations/ceph/?query=973f5d3d606a4cca9b23340cb6decf9e Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi186 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7224687] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:09:46 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224687/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224687/ sentry: https://sentry.ceph.com/organizations/ceph/?query=94f9d3de99544947a72c4acfbbc686c5 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi121 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7224707] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:31:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224707/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224707/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50c9ba0eea514181bbfa8790a02f93b1 Command failed (workunit test rados/test.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7224724] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:26:56 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224724/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224724/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f04f55b3a2804114b440cbebdbb2ffb5 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a2027c62f979d9192a38f096eacd4408d4a43a1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7224728] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224728/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224728/ sentry: https://sentry.ceph.com/organizations/ceph/?query=550dd6fddcca4e0588815f250d5f63af Command failed on smithi003 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' Dead ================================================================= [7224099] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224099/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224099/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [7224627] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224627/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224627/ hit max job timeout Pass ================================================================= [7224041] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224041/ [7224043] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_nfs} time: 00:33:20 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224043/ [7224045] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224045/ [7224047] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224047/ [7224049] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224049/ [7224051] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:33 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224051/ [7224053] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} time: 02:56:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224053/ [7224057] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224057/ [7224059] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:14:15 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224059/ [7224061] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224061/ [7224063] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:29:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224063/ [7224065] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:13:32 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224065/ [7224067] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:16 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224067/ [7224069] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224069/ [7224071] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:13:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224071/ [7224073] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:44:36 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224073/ [7224075] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:29:15 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224075/ [7224077] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224077/ [7224079] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224079/ [7224081] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:34:32 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224081/ [7224083] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224083/ [7224085] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224085/ [7224087] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:20:17 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224087/ [7224089] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224089/ [7224091] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:24:12 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224091/ [7224093] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli} time: 00:16:58 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224093/ [7224095] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_8}} time: 00:11:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224095/ [7224097] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:26:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224097/ [7224101] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:42:45 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224101/ [7224103] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224103/ [7224105] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224105/ [7224107] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224107/ [7224109] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224109/ [7224111] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/libcephsqlite} time: 00:16:11 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224111/ [7224113] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:52 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224113/ [7224115] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub} time: 02:47:06 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224115/ [7224117] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:05:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224117/ [7224119] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_20.04} tasks/workunits} time: 00:10:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224119/ [7224121] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:45 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224121/ [7224123] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_orch_cli_mon} time: 00:25:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224123/ [7224125] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:40:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224125/ [7224127] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224127/ [7224129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224129/ [7224131] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224131/ [7224133] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224133/ [7224135] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:25 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224135/ [7224137] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:10:02 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224137/ [7224139] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:31:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224139/ [7224141] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:18:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224141/ [7224143] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224143/ [7224145] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224145/ [7224147] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:32:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224147/ [7224149] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:30:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224149/ [7224151] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224151/ [7224153] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224153/ [7224157] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224157/ [7224159] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:21:34 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224159/ [7224161] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:32:59 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224161/ [7224163] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224163/ [7224165] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:27:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224165/ [7224166] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:37:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224166/ [7224168] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:33 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224168/ [7224170] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224170/ [7224172] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224172/ [7224176] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224176/ [7224178] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:01:52 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224178/ [7224180] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_20.04} tasks/crash} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224180/ [7224182] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_20.04}} time: 00:40:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224182/ [7224184] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:16:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224184/ [7224188] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224188/ [7224190] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/c2c} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224190/ [7224192] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:26:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224192/ [7224194] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224194/ [7224196] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224196/ [7224200] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224200/ [7224203] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:16:52 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224203/ [7224206] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:37:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224206/ [7224208] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224208/ [7224211] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224211/ [7224214] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224214/ [7224216] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects} time: 00:38:17 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224216/ [7224219] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:19:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224219/ [7224222] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:16:40 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224222/ [7224225] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224225/ [7224227] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews} time: 00:09:11 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224227/ [7224230] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench} time: 00:30:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224230/ [7224233] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224233/ [7224235] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:15 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224235/ [7224238] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224238/ [7224241] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224241/ [7224244] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224244/ [7224246] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224246/ [7224249] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:30:40 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224249/ [7224252] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:07:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224252/ [7224254] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:18 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224254/ [7224257] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224257/ [7224260] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224260/ [7224262] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224262/ [7224265] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_20.04} tasks/failover} time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224265/ [7224268] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:08:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224268/ [7224271] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:24:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224271/ [7224273] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224273/ [7224276] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:32:49 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224276/ [7224282] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224282/ [7224284] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:17:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224284/ [7224287] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224287/ [7224290] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:18:52 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224290/ [7224292] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224292/ [7224295] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224295/ [7224298] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:13:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224298/ [7224300] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:12:49 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224300/ [7224303] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:12:06 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224303/ [7224306] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:21:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224306/ [7224309] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224309/ [7224311] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:25:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224311/ [7224314] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:45 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224314/ [7224317] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:19:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224317/ [7224319] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:11:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224319/ [7224322] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224322/ [7224325] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224325/ [7224327] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:18:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224327/ [7224330] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224330/ [7224333] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:25:19 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224333/ [7224336] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224336/ [7224338] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:31:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224338/ [7224341] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:21:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224341/ [7224344] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} time: 00:29:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224344/ [7224346] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:39 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224346/ [7224349] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:57 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224349/ [7224352] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_20.04} tasks/insights} time: 00:12:19 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224352/ [7224355] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:35:36 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224355/ [7224357] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224357/ [7224360] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:48 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224360/ [7224363] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224363/ [7224365] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 01:28:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224365/ [7224367] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:21:45 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224367/ [7224370] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:24:11 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224370/ [7224373] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:56 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224373/ [7224376] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224376/ [7224379] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224379/ [7224381] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224381/ [7224390] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224390/ [7224392] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:46:40 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224392/ [7224395] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:24:46 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224395/ [7224398] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:15:15 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224398/ [7224400] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} time: 00:18:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224400/ [7224403] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_striper} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224403/ [7224406] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224406/ [7224409] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:27 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224409/ [7224411] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224411/ [7224414] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:32:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224414/ [7224417] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224417/ [7224419] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224419/ [7224422] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224422/ [7224425] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224425/ [7224427] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:38:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224427/ [7224429] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224429/ [7224433] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224433/ [7224436] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:22 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224436/ [7224438] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:59 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224438/ [7224441] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:11:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224441/ [7224444] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:25:30 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224444/ [7224447] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224447/ [7224449] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224449/ [7224452] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:32 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224452/ [7224454] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} time: 00:21:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224454/ [7224456] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224456/ [7224459] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:30:12 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224459/ [7224462] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:34:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224462/ [7224465] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224465/ [7224468] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:30:23 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224468/ [7224471] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:20:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224471/ [7224473] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:33:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224473/ [7224476] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224476/ [7224479] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:26:13 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224479/ [7224484] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:26:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224484/ [7224486] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224486/ [7224489] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:10:06 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224489/ [7224493] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:25:16 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224493/ [7224495] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:07 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224495/ [7224498] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:23:18 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224498/ [7224501] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224501/ [7224503] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224503/ [7224509] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:17:25 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224509/ [7224511] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224511/ [7224514] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224514/ [7224517] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224517/ [7224520] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224520/ [7224522] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:36:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224522/ [7224525] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224525/ [7224528] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:25:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224528/ [7224530] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:28:17 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224530/ [7224533] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:23:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224533/ [7224535] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:40:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224535/ [7224537] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} time: 00:17:45 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224537/ [7224540] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:31:18 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224540/ [7224548] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:12:35 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224548/ [7224552] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224552/ [7224555] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:40 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224555/ [7224557] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224557/ [7224560] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224560/ [7224563] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:37:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224563/ [7224565] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:01 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224565/ [7224568] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224568/ [7224571] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224571/ [7224574] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:42 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224574/ [7224575] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:00:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224575/ [7224582] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224582/ [7224583] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:15:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224583/ [7224586] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:28:55 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224586/ [7224589] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} tasks/dashboard} time: 01:01:38 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224589/ [7224594] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224594/ [7224597] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:36:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224597/ [7224600] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:09:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224600/ [7224601] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:29:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224601/ [7224603] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:10:57 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224603/ [7224607] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:13:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224607/ [7224608] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:43 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224608/ [7224610] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224610/ [7224612] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:22 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224612/ [7224613] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224613/ [7224615] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:36:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224615/ [7224617] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224617/ [7224618] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:57:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224618/ [7224620] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224620/ [7224622] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:24:36 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224622/ [7224623] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224623/ [7224625] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224625/ [7224629] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:08:56 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224629/ [7224630] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224630/ [7224632] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:35:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224632/ [7224634] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} time: 02:14:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224634/ [7224635] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:22:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224635/ [7224637] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/readwrite} time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224637/ [7224639] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224639/ [7224641] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224641/ [7224642] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:17:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224642/ [7224644] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits} time: 00:30:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224644/ [7224646] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224646/ [7224648] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224648/ [7224649] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:13:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224649/ [7224651] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:30:37 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224651/ [7224653] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224653/ [7224654] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:49 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224654/ [7224656] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:24:00 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224656/ [7224658] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224658/ [7224659] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224659/ [7224661] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224661/ [7224663] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:27:02 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224663/ [7224664] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224664/ [7224666] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224666/ [7224668] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:12:10 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224668/ [7224670] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:18:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224670/ [7224671] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:55 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224671/ [7224673] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224673/ [7224675] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224675/ [7224677] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224677/ [7224678] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:15 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224678/ [7224680] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:29 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224680/ [7224682] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/osd-backfill} time: 03:33:16 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224682/ [7224683] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:22:22 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224683/ [7224685] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:15:51 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224685/ [7224689] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224689/ [7224690] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:19:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224690/ [7224692] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224692/ [7224694] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:11:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224694/ [7224695] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224695/ [7224697] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:24:50 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224697/ [7224699] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:36:14 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224699/ [7224700] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:20:08 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224700/ [7224702] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{rhel_8}} time: 00:12:09 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224702/ [7224704] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224704/ [7224705] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224705/ [7224709] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:34:26 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224709/ [7224711] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224711/ [7224712] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224712/ [7224714] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224714/ [7224716] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:25:46 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224716/ [7224717] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224717/ [7224719] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224719/ [7224721] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:19 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224721/ [7224723] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:24:44 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224723/ [7224726] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/crash} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224726/ [7224729] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:32 info: http://pulpito.ceph.com/yuriw-2023-03-28_22:45:07-rados-wip-yuri7-testing-2023-03-28-0942-distro-default-smithi/7224729/