2022-10-14T09:09:13.686 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-14T09:09:13.687 INFO:teuthology.results:subset: '111/120000' 2022-10-14T09:09:13.687 INFO:teuthology.results:seed: '98' 2022-10-14T12:31:13.009 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-10-14T12:31:13.041 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-10-14T16:24:34.042 INFO:teuthology.results:Tests finished! gathering results... 2022-10-14T16:24:59.792 INFO:scrape:Found 320 jobs 2022-10-14T16:25:05.697 INFO:scrape:Found 8 distinct failure reasons 2022-10-14T16:25:05.698 INFO:scrape:Failure: Command failed on smithi120 with status 1: 'sudo yum -y install ceph-radosgw' 2022-10-14T16:25:05.699 INFO:scrape:76 jobs 2022-10-14T16:25:05.700 INFO:scrape:suites intersection: [] 2022-10-14T16:25:05.700 INFO:scrape:suites union: ['0-start', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '1-tasks', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'agent/off', 'agent/on', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/3-mons', 'clusters/9-mons', 'clusters/{fixed-2', 'clusters/{fixed-4', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'fast/fast', 'fast/normal', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/mon-delay', 'msgr-failures/none', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/monthrash/{ceph', 'rados/multimon/{clusters/3', 'rados/multimon/{clusters/9', 'rados/objectstore/{backends/filejournal', 'rados/objectstore/{backends/keyvaluedb', 'rados/objectstore/{backends/objectstore-filestore-memstore', 'rados/singleton-nomsgr/{all/admin_socket_output', 'rados/singleton-nomsgr/{all/ceph-kvstore-tool', 'rados/singleton-nomsgr/{all/ceph-post-file', 'rados/singleton-nomsgr/{all/crushdiff', 'rados/singleton-nomsgr/{all/export-after-evict', 'rados/singleton-nomsgr/{all/full-tiering', 'rados/singleton-nomsgr/{all/large-omap-object-warnings', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/singleton-nomsgr/{all/osd_stale_reads', 'rados/singleton-nomsgr/{all/pool-access', 'rados/singleton-nomsgr/{all/recovery-unfound-found', 'rados/singleton-nomsgr/{all/version-number-sanity', 'rados/singleton/{all/lost-unfound-delete', 'rados/singleton/{all/mon-config-key-caps', 'rados/singleton/{all/mon-config-keys', 'rados/singleton/{all/osd-recovery', 'rados/singleton/{all/recovery-preemption', 'rados/singleton/{all/test-crash', 'rados/singleton/{all/test_envlibrados_for_rocksdb', 'rados/singleton/{all/thrash-rados/{thrash-rados', 'rados/standalone/{supported-random-distro$/{rhel_8}', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash-erasure-code-shec/{ceph', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'task/test_adoption}', 'task/test_cephadm}', 'task/test_iscsi_pids_limit/{centos_8.stream_container_tools', 'task/test_nfs}', 'task/test_orch_cli_mon}', 'task/test_orch_cli}', 'tasks/mon_clock_no_skews}', 'tasks/mon_recovery}', 'test_iscsi_pids_limit}}', 'test_rbd_api', 'test_rbd_python}}', 'thrashers/careful', 'thrashers/default', 'thrashers/fastread', 'thrashers/force-sync-many', 'thrashers/many', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/one', 'thrashers/pggrow', 'thrashers/sync', 'thrashers/sync-many', 'thrashosds-health', 'thrashosds-health}', 'upgrade-sequence', 'workload/{ec-rados-default', 'workloads/cache-pool-snaps}', 'workloads/cache-snaps-balanced}', 'workloads/ec-rados-plugin=clay-k=4-m=2}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'workloads/ec-small-objects-fast-read}', 'workloads/ec-small-objects-overwrites}', 'workloads/ec-small-objects}', 'workloads/mgr}', 'workloads/mon}', 'workloads/pool-create-delete}', 'workloads/rados_5925}', 'workloads/rados_api_tests}', 'workloads/rados_mon_osdmap_prune}', 'workloads/rados_mon_workunits}', 'workloads/redirect_set_object}', 'workloads/redirect}', 'workloads/set-chunks-read}', 'workloads/small-objects-balanced}', 'workloads/small-objects-localized}', 'workloads/small-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects}', 'workloads/write_fadvise_dontneed}'] 2022-10-14T16:25:05.701 INFO:scrape: 2022-10-14T16:25:05.702 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi111.front.sepia.ceph.com: ['type=AVC msg=audit(1665746296.019:7014): avc: denied { ioctl } for pid=58631 comm="iptables" path="/var/lib/containers/storage/overlay/2d5d241c6e24d75c8391325fc03adb78a1c79f21c8f76f497dafd9ce3c6f68df/merged" dev="overlay" ino=3281441 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-10-14T16:25:05.703 INFO:scrape:1 jobs: ['7065645'] 2022-10-14T16:25:05.703 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-10-14T16:25:05.704 INFO:scrape: 2022-10-14T16:25:05.705 INFO:scrape:Failure: Command failed on smithi132 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-285-g6fb85925-1focal ceph-mds=18.0.0-285-g6fb85925-1focal ceph-common=18.0.0-285-g6fb85925-1focal ceph-fuse=18.0.0-285-g6fb85925-1focal ceph-test=18.0.0-285-g6fb85925-1focal radosgw=18.0.0-285-g6fb85925-1focal python-ceph=18.0.0-285-g6fb85925-1focal libcephfs1=18.0.0-285-g6fb85925-1focal libcephfs-java=18.0.0-285-g6fb85925-1focal libcephfs-jni=18.0.0-285-g6fb85925-1focal librados2=18.0.0-285-g6fb85925-1focal librbd1=18.0.0-285-g6fb85925-1focal rbd-fuse=18.0.0-285-g6fb85925-1focal' 2022-10-14T16:25:05.705 INFO:scrape:1 jobs: ['7065722'] 2022-10-14T16:25:05.706 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_orch_cli}'] 2022-10-14T16:25:05.706 INFO:scrape: 2022-10-14T16:25:05.707 INFO:scrape:Failure: Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi187 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fb859258928a5d76fd5f6b6802b10593037f421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh' 2022-10-14T16:25:05.707 INFO:scrape:1 jobs: ['7065580'] 2022-10-14T16:25:05.708 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/erasure-code}'] 2022-10-14T16:25:05.708 INFO:scrape: 2022-10-14T16:25:05.709 INFO:scrape:Failure: No module named 'tasks.ceph' 2022-10-14T16:25:05.709 INFO:scrape:1 jobs: ['7065721'] 2022-10-14T16:25:05.710 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/none', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2022-10-14T16:25:05.710 INFO:scrape: 2022-10-14T16:25:05.711 INFO:scrape:Failure: 'check osd count' reached maximum tries (90) after waiting for 900 seconds 2022-10-14T16:25:05.711 INFO:scrape:2 jobs: ['7065747', '7065587'] 2022-10-14T16:25:05.712 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', 'cluster/3-node', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-14T16:25:05.712 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-10-14T16:25:05.713 INFO:scrape: 2022-10-14T16:25:05.713 INFO:scrape:Failure: Command failed on smithi112 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel' 2022-10-14T16:25:05.713 INFO:scrape:2 jobs: ['7065723', '7065676'] 2022-10-14T16:25:05.714 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{rhel_8}'] 2022-10-14T16:25:05.714 INFO:scrape:suites union: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{rhel_8}', 'tasks/readwrite}', 'tasks/scrub_test}'] 2022-10-14T16:25:05.715 INFO:scrape: 2022-10-14T16:25:05.715 INFO:scrape:Failure: Command failed on smithi163 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' 2022-10-14T16:25:05.716 INFO:scrape:2 jobs: ['7065507', '7065665'] 2022-10-14T16:25:05.716 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', 'cluster/1-node', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}'] 2022-10-14T16:25:05.717 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', 'cluster/1-node', 'k8s/1.21', 'net/calico', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}'] 2022-10-14T16:25:05.717 INFO:scrape: 2022-10-14T16:25:05.729 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/ failed: 86 dead: 0 running: 0 waiting: 0 queued: 0 passed: 234 Fail ================================================================= [7065475] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:19 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065475/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065475/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7df6065dbbd4433f9e1b6a5731c7b1c9 Command failed on smithi196 with status 1: 'sudo yum -y install ceph- radosgw' [7065476] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_mon_osdmap_prune} ----------------------------------------------------------------- time: 00:07:19 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065476/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065476/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b83fefb29d14505bc9c141d982f8b52 Command failed on smithi134 with status 1: 'sudo yum -y install ceph- radosgw' [7065480] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065480/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065480/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5944e1d82844efbba4c22df4fba021a Command failed on smithi094 with status 1: 'sudo yum -y install ceph- radosgw' [7065485] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065485/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065485/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3e1011a3af8e45f483afb14b34f99478 Command failed on smithi018 with status 1: 'sudo yum -y install ceph- radosgw' [7065486] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065486/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065486/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6d1fdacd17749e183356128b113a596 Command failed on smithi103 with status 1: 'sudo yum -y install ceph- radosgw' [7065487] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065487/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065487/ sentry: https://sentry.ceph.com/organizations/ceph/?query=89aa827174b2467facbda1b6d966764e Command failed on smithi196 with status 1: 'sudo yum -y install ceph- radosgw' [7065490] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} ----------------------------------------------------------------- time: 00:07:34 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065490/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065490/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d189a75fb6a64a4fb10e1255d3ef6432 Command failed on smithi116 with status 1: 'sudo yum -y install ceph- radosgw' [7065496] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:23 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065496/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065496/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52da0cb78a7646b2b520a254ba446812 Command failed on smithi203 with status 1: 'sudo yum -y install ceph- radosgw' [7065507] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:04:29 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065507/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065507/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5eef77ee360646c598bbbd9b10e5117c Command failed on smithi163 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7065508] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:37 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065508/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065508/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b3af1e5d34a43779df10c56620a7dd5 Command failed on smithi052 with status 1: 'sudo yum -y install ceph- radosgw' [7065510] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:08:13 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065510/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065510/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dbd9cbaa0f374fba922fb20c3ee42539 Command failed on smithi080 with status 1: 'sudo yum -y install ceph- radosgw' [7065512] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:08:31 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065512/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065512/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bcd581fbad404f4a99eaf6da0b7603cb Command failed on smithi055 with status 1: 'sudo yum -y install ceph- radosgw' [7065514] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065514/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065514/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6434fc65c0e4a10af650126cb23eaba Command failed on smithi087 with status 1: 'sudo yum -y install ceph- radosgw' [7065522] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} ----------------------------------------------------------------- time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065522/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065522/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f39266515d38458182c264b4e1f29628 Command failed on smithi088 with status 1: 'sudo yum -y install ceph- radosgw' [7065534] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} ----------------------------------------------------------------- time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065534/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065534/ sentry: https://sentry.ceph.com/organizations/ceph/?query=71038416cc194110b4d5a08c3ef75d54 Command failed on smithi196 with status 1: 'sudo yum -y install ceph- radosgw' [7065537] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} ----------------------------------------------------------------- time: 00:07:39 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065537/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065537/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a751a8fbd90f49c0990826f05e05c33b Command failed on smithi161 with status 1: 'sudo yum -y install ceph- radosgw' [7065538] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065538/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065538/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12d7cad44f3f45ae81ab78959a8818c8 Command failed on smithi178 with status 1: 'sudo yum -y install ceph- radosgw' [7065541] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 00:07:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065541/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065541/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d96ae319a2834b458a9ab45dfd30871c Command failed on smithi017 with status 1: 'sudo yum -y install ceph- radosgw' [7065544] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065544/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065544/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d930e87d2d974bcaa799b03ed84caca2 Command failed on smithi145 with status 1: 'sudo yum -y install ceph- radosgw' [7065548] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065548/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065548/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06167884c37a44ab83c56e3dedcd437a Command failed on smithi179 with status 1: 'sudo yum -y install ceph- radosgw' [7065550] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:07:40 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065550/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065550/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0b4fafe7fcb0401398cebf91d55be94f Command failed on smithi088 with status 1: 'sudo yum -y install ceph- radosgw' [7065555] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:34 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065555/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065555/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fcbe4fbf7ce348c68ba362fff94bb404 Command failed on smithi138 with status 1: 'sudo yum -y install ceph- radosgw' [7065559] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065559/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065559/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eba549acc16f428d9a03ebb5d2878eac Command failed on smithi171 with status 1: 'sudo yum -y install ceph- radosgw' [7065564] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:53 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065564/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065564/ sentry: https://sentry.ceph.com/organizations/ceph/?query=35640163704749b2b97704e5f32cc8cc Command failed on smithi103 with status 1: 'sudo yum -y install ceph- radosgw' [7065565] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:08:04 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065565/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065565/ sentry: https://sentry.ceph.com/organizations/ceph/?query=671151be0eef4761a856c044ac24cc60 Command failed on smithi102 with status 1: 'sudo yum -y install ceph- radosgw' [7065567] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065567/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065567/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f426f0d964a4728984dc6e115c5845a Command failed on smithi155 with status 1: 'sudo yum -y install ceph- radosgw' [7065568] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} ----------------------------------------------------------------- time: 00:08:00 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065568/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065568/ sentry: https://sentry.ceph.com/organizations/ceph/?query=abd201c786284d80a7cc442512b2a8bf Command failed on smithi143 with status 1: 'sudo yum -y install ceph- radosgw' [7065570] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065570/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065570/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3d824f659a9e41429032a5785bddbc68 Command failed on smithi159 with status 1: 'sudo yum -y install ceph- radosgw' [7065572] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/one workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:07:59 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065572/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065572/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1fa5382c8265456abfc9ca4dd3597141 Command failed on smithi099 with status 1: 'sudo yum -y install ceph- radosgw' [7065573] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} ----------------------------------------------------------------- time: 00:07:50 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065573/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065573/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e80b67d56dae4f9287a2b5e678d6d192 Command failed on smithi115 with status 1: 'sudo yum -y install ceph- radosgw' [7065580] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} ----------------------------------------------------------------- time: 00:22:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065580/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065580/ sentry: https://sentry.ceph.com/organizations/ceph/?query=265d8db832464e0b91ba5b0a21993805 Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi187 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fb859258928a5d76fd5f6b6802b10593037f421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test- erasure-code.sh' [7065587] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:25:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065587/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065587/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48c17fca798545999f6bbcfa259ecbde 'check osd count' reached maximum tries (90) after waiting for 900 seconds [7065588] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:23 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065588/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065588/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67b3b0e3ebce4802b2c3f6352e601f5a Command failed on smithi120 with status 1: 'sudo yum -y install ceph- radosgw' [7065592] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:08:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065592/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065592/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cf1f1d28b0e843e8b2222ccf14ff2c4d Command failed on smithi043 with status 1: 'sudo yum -y install ceph- radosgw' [7065596] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065596/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065596/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6187c8c3f39f4b05b35cd77a132e880f Command failed on smithi093 with status 1: 'sudo yum -y install ceph- radosgw' [7065599] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_recovery} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065599/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065599/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c611eb5f63ad4cb0a102ff8f98b09024 Command failed on smithi115 with status 1: 'sudo yum -y install ceph- radosgw' [7065607] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} ----------------------------------------------------------------- time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065607/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065607/ sentry: https://sentry.ceph.com/organizations/ceph/?query=51520379f54f4170bb2a266014b0635b Command failed on smithi094 with status 1: 'sudo yum -y install ceph- radosgw' [7065608] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:07:56 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065608/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065608/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20ca3d5b76d8401199e8583e9662446d Command failed on smithi188 with status 1: 'sudo yum -y install ceph- radosgw' [7065612] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:09:01 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065612/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065612/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e668b40b8ec44dd8e4b19aa9be9f993 Command failed on smithi197 with status 1: 'sudo yum -y install ceph- radosgw' [7065615] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} ----------------------------------------------------------------- time: 00:07:35 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065615/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065615/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ce8408a1460a4f8ea9e0da915ba33c1c Command failed on smithi182 with status 1: 'sudo yum -y install ceph- radosgw' [7065618] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} ----------------------------------------------------------------- time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065618/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065618/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ffb51f7c39294afa99cf3024b1af09b6 Command failed on smithi063 with status 1: 'sudo yum -y install ceph- radosgw' [7065624] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} ----------------------------------------------------------------- time: 00:07:39 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065624/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065624/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c4652e1b0e4b435d88fd9b3756a0ce8a Command failed on smithi134 with status 1: 'sudo yum -y install ceph- radosgw' [7065625] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:07:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065625/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065625/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1d0a7580ec554770a7c69d91723fced2 Command failed on smithi115 with status 1: 'sudo yum -y install ceph- radosgw' [7065637] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065637/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065637/ sentry: https://sentry.ceph.com/organizations/ceph/?query=866916e6977c4554aafd3641071036a1 Command failed on smithi197 with status 1: 'sudo yum -y install ceph- radosgw' [7065638] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} ----------------------------------------------------------------- time: 00:07:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065638/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065638/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5673a50f6e18400c91d84b44fbdd5805 Command failed on smithi017 with status 1: 'sudo yum -y install ceph- radosgw' [7065643] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:07:56 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065643/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065643/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e1834c7d2a394345adb153b806d3955d Command failed on smithi153 with status 1: 'sudo yum -y install ceph- radosgw' [7065645] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065645/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065645/ SELinux denials found on ubuntu@smithi111.front.sepia.ceph.com: ['type=AVC msg=audit(1665746296.019:7014): avc: denied { ioctl } for pid=58631 comm="iptables" path="/var/lib/containers/storage/overlay/2d5d241c6e24d75c8 391325fc03adb78a1c79f21c8f76f497dafd9ce3c6f68df/merged" dev="overlay" ino=3281441 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7065651] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} ----------------------------------------------------------------- time: 00:07:31 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065651/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065651/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf9ac5b45542477ea7647e2c23ace53b Command failed on smithi154 with status 1: 'sudo yum -y install ceph- radosgw' [7065656] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:07:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065656/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065656/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e089885a443d48228818436baa76a3ff Command failed on smithi107 with status 1: 'sudo yum -y install ceph- radosgw' [7065665] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:04:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065665/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065665/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c17c11650034cc89ce605bb9dd5b1f5 Command failed on smithi112 with status 1: 'kubectl create -f rook/cluster/examples/kubernetes/ceph/crds.yaml -f rook/cluster/examples/kubernetes/ceph/common.yaml -f operator.yaml' [7065666] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065666/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065666/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6bfd6e9c7fde491b9e38dc47be622145 Command failed on smithi163 with status 1: 'sudo yum -y install ceph- radosgw' [7065667] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:08:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065667/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065667/ sentry: https://sentry.ceph.com/organizations/ceph/?query=004d29f818dc490c9441781a3265751f Command failed on smithi145 with status 1: 'sudo yum -y install ceph- radosgw' [7065672] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065672/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065672/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1fd21acf3aa64f74bd15003338a1c53d Command failed on smithi120 with status 1: 'sudo yum -y install ceph- radosgw' [7065676] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/readwrite} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065676/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065676/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee7e1015f05f4671be88fd6faf09b47c Command failed on smithi111 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite- devel' [7065683] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:19 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065683/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065683/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad748f912b024a83888b703dfc26ef75 Command failed on smithi171 with status 1: 'sudo yum -y install ceph- radosgw' [7065687] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} ----------------------------------------------------------------- time: 00:07:17 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065687/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065687/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d4e5cf3b0b1c48d09c2ceeca653bd89e Command failed on smithi134 with status 1: 'sudo yum -y install ceph- radosgw' [7065692] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065692/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065692/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ceb3243417447db933173ff85716e86 Command failed on smithi061 with status 1: 'sudo yum -y install ceph- radosgw' [7065693] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065693/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065693/ sentry: https://sentry.ceph.com/organizations/ceph/?query=64244e093871436ca2a32937899bc95c Command failed on smithi164 with status 1: 'sudo yum -y install ceph- radosgw' [7065695] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} ----------------------------------------------------------------- time: 00:03:12 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065695/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065695/ sentry: https://sentry.ceph.com/organizations/ceph/?query=698a394ef79047f4a97a900c1c77ae6e Command failed on smithi179 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' [7065698] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} ----------------------------------------------------------------- time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065698/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b8808d2a1e8648b88c7d8bb0810116e4 Command failed on smithi120 with status 1: 'sudo yum -y install ceph- radosgw' [7065702] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065702/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065702/ sentry: https://sentry.ceph.com/organizations/ceph/?query=222f8658e99d4d7394a3ca4161476504 Command failed on smithi133 with status 1: 'sudo yum -y install ceph- radosgw' [7065703] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065703/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065703/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea5378a047e6418d9531f750d1ea76f2 Command failed on smithi106 with status 1: 'sudo yum -y install ceph- radosgw' [7065704] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065704/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065704/ sentry: https://sentry.ceph.com/organizations/ceph/?query=88ed753f34b04000b1739d269ca77c0d Command failed on smithi093 with status 1: 'sudo yum -y install ceph- radosgw' [7065711] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065711/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065711/ sentry: https://sentry.ceph.com/organizations/ceph/?query=865f13280fd2491fb60bd6c42ea94c70 Command failed on smithi171 with status 1: 'sudo yum -y install ceph- radosgw' [7065713] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065713/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065713/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d01c7ff42005488aa13c2b1022fcf961 Command failed on smithi134 with status 1: 'sudo yum -y install ceph- radosgw' [7065714] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} ----------------------------------------------------------------- time: 00:07:32 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065714/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065714/ sentry: https://sentry.ceph.com/organizations/ceph/?query=87e287edfdff4610913e7ebddb43e960 Command failed on smithi115 with status 1: 'sudo yum -y install ceph- radosgw' [7065718] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065718/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065718/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03d04ce0bd3c44d9bbf29245d17b6817 Command failed on smithi033 with status 1: 'sudo yum -y install ceph- radosgw' [7065721] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:06:14 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065721/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065721/ sentry: https://sentry.ceph.com/organizations/ceph/?query=71051588dae04c6eb30f4defb8b1c45e No module named 'tasks.ceph' [7065722] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} ----------------------------------------------------------------- time: 00:02:34 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065722/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065722/ sentry: https://sentry.ceph.com/organizations/ceph/?query=844bac68a5f9454689d7d7c6822b8c4c Command failed on smithi132 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::=" --force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-285-g6fb85925-1focal ceph-mds=18.0.0-285-g6fb85925-1focal ceph- common=18.0.0-285-g6fb85925-1focal ceph-fuse=18.0.0-285-g6fb85925-1focal ceph-test=18.0.0-285-g6fb85925-1focal radosgw=18.0.0-285-g6fb85925-1focal python-ceph=18.0.0-285-g6fb85925-1focal libcephfs1=18.0.0-285-g6fb85925-1focal libcephfs- java=18.0.0-285-g6fb85925-1focal libcephfs-jni=18.0.0-285-g6fb85925-1focal librados2=18.0.0-285-g6fb85925-1focal librbd1=18.0.0-285-g6fb85925-1focal rbd-fuse=18.0.0-285-g6fb85925-1focal' [7065723] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/scrub_test} ----------------------------------------------------------------- time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065723/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065723/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc29908ed96740739136a586f1d0a149 Command failed on smithi112 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite- devel' [7065728] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065728/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065728/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0bd1a4b9ddcb437e80c19a8b773b8ab9 Command failed on smithi106 with status 1: 'sudo yum -y install ceph- radosgw' [7065732] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} ----------------------------------------------------------------- time: 00:07:37 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065732/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065732/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2fe21963529a44cb9b6c63b53aad294b Command failed on smithi134 with status 1: 'sudo yum -y install ceph- radosgw' [7065740] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065740/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065740/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4b4674de90c4e43a984ce308c76f77a Command failed on smithi150 with status 1: 'sudo yum -y install ceph- radosgw' [7065742] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065742/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065742/ sentry: https://sentry.ceph.com/organizations/ceph/?query=79ae7645e1034532b7c1bcd07af71b4a Command failed on smithi188 with status 1: 'sudo yum -y install ceph- radosgw' [7065744] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065744/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065744/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dfcee1a86fa245edbe781e511ba3b5ec Command failed on smithi026 with status 1: 'sudo yum -y install ceph- radosgw' [7065747] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:25:15 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065747/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065747/ sentry: https://sentry.ceph.com/organizations/ceph/?query=94d2daf2b7c24aa086069ef479b398d8 'check osd count' reached maximum tries (90) after waiting for 900 seconds [7065750] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065750/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065750/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0de918006b91441aaddab39236838a40 Command failed on smithi197 with status 1: 'sudo yum -y install ceph- radosgw' [7065753] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065753/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065753/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c87e9819d5ad46afa4c33ef1fc258bb3 Command failed on smithi047 with status 1: 'sudo yum -y install ceph- radosgw' [7065755] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065755/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065755/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7db43de93c6b49efbd9e5e6a82f74a30 Command failed on smithi120 with status 1: 'sudo yum -y install ceph- radosgw' [7065757] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065757/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065757/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48be2a940d3d48a496ba05c53e37dfdb Command failed on smithi149 with status 1: 'sudo yum -y install ceph- radosgw' [7065762] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:07:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065762/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065762/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fc662a88182842aa81812510cf703159 Command failed on smithi093 with status 1: 'sudo yum -y install ceph- radosgw' [7065779] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} ----------------------------------------------------------------- time: 00:07:31 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065779/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065779/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48cc0e3e3bdd44df87483893c7320511 Command failed on smithi103 with status 1: 'sudo yum -y install ceph- radosgw' [7065782] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065782/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065782/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fb77e721a25b45648c20fe93e93bf2ce Command failed on smithi093 with status 1: 'sudo yum -y install ceph- radosgw' [7065784] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:07:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065784/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065784/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2a9c741063f417abe72b5dce3ef4bcd Command failed on smithi107 with status 1: 'sudo yum -y install ceph- radosgw' [7065785] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:08:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065785/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065785/ sentry: https://sentry.ceph.com/organizations/ceph/?query=13b496ffca9541f7a12eb66303b660cb Command failed on smithi049 with status 1: 'sudo yum -y install ceph- radosgw' [7065789] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065789/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065789/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b3958599428a4c768259ec715c2aa5d3 Command failed on smithi157 with status 1: 'sudo yum -y install ceph- radosgw' Pass ================================================================= [7065470] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:06:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065470/ [7065471] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:03 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065471/ [7065472] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:20:17 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065472/ [7065473] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_orch_cli_mon} time: 00:22:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065473/ [7065474] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:22:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065474/ [7065477] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:24:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065477/ [7065478] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:28:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065478/ [7065479] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:24:11 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065479/ [7065481] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065481/ [7065482] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:08:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065482/ [7065483] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065483/ [7065484] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:59:53 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065484/ [7065488] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065488/ [7065489] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:37:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065489/ [7065491] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:22:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065491/ [7065492] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:07:53 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065492/ [7065493] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:18:14 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065493/ [7065494] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065494/ [7065495] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:44 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065495/ [7065497] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:29:37 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065497/ [7065498] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065498/ [7065499] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065499/ [7065500] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:16:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065500/ [7065501] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:22:45 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065501/ [7065502] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:09 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065502/ [7065503] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} tasks/dashboard} time: 01:13:48 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065503/ [7065504] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{filestore-xfs} supported-random-distro$/{centos_8} tasks/crash} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065504/ [7065505] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:06:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065505/ [7065506] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065506/ [7065509] rados/standalone/{supported-random-distro$/{centos_8} workloads/c2c} time: 00:12:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065509/ [7065511] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065511/ [7065513] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:18:42 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065513/ [7065515] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:23:07 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065515/ [7065516] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065516/ [7065517] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:22:13 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065517/ [7065518] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065518/ [7065519] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:48 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065519/ [7065520] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:56 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065520/ [7065521] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:06:04 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065521/ [7065523] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065523/ [7065524] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065524/ [7065525] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:23:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065525/ [7065526] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:27:36 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065526/ [7065527] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065527/ [7065528] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065528/ [7065529] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:12:48 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065529/ [7065530] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:11:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065530/ [7065531] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065531/ [7065532] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065532/ [7065533] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:40 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065533/ [7065535] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:10:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065535/ [7065536] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:03:51 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065536/ [7065539] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:06:07 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065539/ [7065540] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/failover} time: 00:13:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065540/ [7065542] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:26 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065542/ [7065543] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065543/ [7065545] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:28:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065545/ [7065546] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:51:29 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065546/ [7065547] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065547/ [7065549] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065549/ [7065551] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}} time: 00:12:09 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065551/ [7065552] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065552/ [7065553] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065553/ [7065554] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065554/ [7065556] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:26:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065556/ [7065557] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065557/ [7065558] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:17 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065558/ [7065560] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:06:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065560/ [7065561] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:01 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065561/ [7065562] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:42 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065562/ [7065563] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065563/ [7065566] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:06:00 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065566/ [7065569] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} time: 00:25:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065569/ [7065571] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:16:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065571/ [7065574] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/insights} time: 00:13:56 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065574/ [7065575] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:08 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065575/ [7065576] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065576/ [7065577] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:30:41 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065577/ [7065578] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065578/ [7065579] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065579/ [7065581] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 01:12:59 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065581/ [7065582] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065582/ [7065583] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:43:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065583/ [7065584] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:06:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065584/ [7065585] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065585/ [7065586] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{filestore-xfs} tasks/e2e} time: 00:19:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065586/ [7065589] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 00:53:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065589/ [7065590] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:33:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065590/ [7065591] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:48:39 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065591/ [7065593] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:18:31 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065593/ [7065594] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:45 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065594/ [7065595] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} time: 02:24:50 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065595/ [7065597] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065597/ [7065598] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065598/ [7065600] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:11:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065600/ [7065601] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:27:38 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065601/ [7065602] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:20:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065602/ [7065603] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:38 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065603/ [7065604] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065604/ [7065605] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:26:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065605/ [7065606] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:06:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065606/ [7065609] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065609/ [7065610] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:28:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065610/ [7065611] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:07:36 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065611/ [7065613] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:59 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065613/ [7065614] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065614/ [7065616] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:17:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065616/ [7065617] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:30:56 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065617/ [7065619] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:21:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065619/ [7065620] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:19:13 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065620/ [7065621] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:07:08 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065621/ [7065622] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065622/ [7065623] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:30:40 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065623/ [7065626] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065626/ [7065627] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:07:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065627/ [7065628] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:28 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065628/ [7065629] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:36:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065629/ [7065630] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:16:11 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065630/ [7065631] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065631/ [7065632] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065632/ [7065633] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:19:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065633/ [7065634] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:21:41 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065634/ [7065635] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:09:39 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065635/ [7065636] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:23:12 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065636/ [7065639] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065639/ [7065640] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065640/ [7065641] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065641/ [7065642] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{centos_8} tasks/progress} time: 00:23:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065642/ [7065644] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:07:14 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065644/ [7065646] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:27:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065646/ [7065647] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:59:51 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065647/ [7065648] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-mixed} time: 00:08:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065648/ [7065649] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:27:38 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065649/ [7065650] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065650/ [7065652] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:05:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065652/ [7065653] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065653/ [7065654] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:10:48 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065654/ [7065655] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:29:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065655/ [7065657] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} time: 00:12:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065657/ [7065658] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065658/ [7065659] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:19:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065659/ [7065660] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065660/ [7065661] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:09:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065661/ [7065662] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:06:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065662/ [7065663] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065663/ [7065664] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:11:50 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065664/ [7065668] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065668/ [7065669] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:17:50 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065669/ [7065670] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:15:38 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065670/ [7065671] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:50 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065671/ [7065673] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:06:15 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065673/ [7065674] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:32:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065674/ [7065675] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:09:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065675/ [7065677] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065677/ [7065678] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:26 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065678/ [7065679] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:58:16 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065679/ [7065680] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065680/ [7065681] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:39:52 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065681/ [7065682] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:11:36 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065682/ [7065684] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:33:10 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065684/ [7065685] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:22:22 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065685/ [7065686] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065686/ [7065688] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:05:19 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065688/ [7065689] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065689/ [7065690] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065690/ [7065691] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:37:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065691/ [7065694] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:08:01 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065694/ [7065696] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065696/ [7065697] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:14:15 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065697/ [7065699] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:51 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065699/ [7065700] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065700/ [7065701] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:58:36 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065701/ [7065705] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:19:33 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065705/ [7065706] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065706/ [7065707] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:32 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065707/ [7065708] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:10:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065708/ [7065709] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065709/ [7065710] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:27:00 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065710/ [7065712] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:07:04 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065712/ [7065715] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065715/ [7065716] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065716/ [7065717] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:10:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065717/ [7065719] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 04:28:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065719/ [7065720] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:17:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065720/ [7065724] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:11:14 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065724/ [7065725] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:16:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065725/ [7065726] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:09:17 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065726/ [7065727] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:07:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065727/ [7065729] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:22:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065729/ [7065730] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:42 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065730/ [7065731] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:31:40 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065731/ [7065733] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:49:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065733/ [7065734] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} time: 00:27:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065734/ [7065735] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065735/ [7065736] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065736/ [7065737] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:25 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065737/ [7065738] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:14 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065738/ [7065739] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} time: 02:12:34 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065739/ [7065741] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:19:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065741/ [7065743] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:23:32 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065743/ [7065745] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e} time: 00:20:18 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065745/ [7065746] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:09:30 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065746/ [7065748] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:41 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065748/ [7065749] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:14:32 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065749/ [7065751] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:22:44 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065751/ [7065752] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:13:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065752/ [7065754] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:58 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065754/ [7065756] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:15:59 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065756/ [7065758] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065758/ [7065759] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:07:15 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065759/ [7065760] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:05:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065760/ [7065761] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065761/ [7065763] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065763/ [7065764] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065764/ [7065765] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:13 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065765/ [7065766] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:01 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065766/ [7065767] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} time: 00:26:02 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065767/ [7065768] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:39:49 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065768/ [7065769] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} time: 02:18:24 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065769/ [7065770] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:27:45 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065770/ [7065771] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_adoption} time: 00:08:43 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065771/ [7065772] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:57 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065772/ [7065773] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:25:27 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065773/ [7065774] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065774/ [7065775] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065775/ [7065776] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:34:13 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065776/ [7065777] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:12:48 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065777/ [7065778] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065778/ [7065780] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:11:20 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065780/ [7065781] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:19 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065781/ [7065783] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:07:44 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065783/ [7065786] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:01 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065786/ [7065787] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065787/ [7065788] rados/singleton/{all/peer mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:06:06 info: http://pulpito.ceph.com/yuriw-2022-10-13_17:24:48-rados-main-distro-default-smithi/7065788/