2021-12-07T07:15:45.557 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-07T07:15:45.560 INFO:teuthology.results:subset: '111/120000' 2021-12-07T07:15:45.561 INFO:teuthology.results:seed: '1645' 2021-12-07T13:20:18.918 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-07T13:20:18.940 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-12-08T01:09:30.034 INFO:teuthology.results:Tests finished! gathering results... 2021-12-08T01:10:10.833 INFO:scrape:Found 427 jobs 2021-12-08T01:10:19.816 INFO:scrape:Found 12 distinct failure reasons 2021-12-08T01:10:19.817 INFO:scrape:Failure: No module named 'tasks.radosbench' 2021-12-08T01:10:19.818 INFO:scrape:7 jobs: ['6549086', '6549128', '6549104', '6549116', '6549133', '6549098', '6549100'] 2021-12-08T01:10:19.818 INFO:scrape:suites intersection: [] 2021-12-08T01:10:19.819 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-pg-log-overrides/normal_pg_log', '1-rook', '1-start', '2-ops/rm-zap-flag}', '2-recovery-overrides/{more-active-recovery}', '2-services/nfs-ingress-rgw-bucket', '2-workload/radosbench', '3-final', '3-final}', '3-scrub-overrides/{default}', 'backoff/peering', 'ceph', 'cluster/1-node', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'k8s/1.21', 'mon_election/classic', 'msgr-failures/few', 'msgr-failures/none', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'net/host', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/singleton-nomsgr/{all/balancer', 'rados/singleton/{all/ec-lost-unfound', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rook/1.7.0}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/repair_test}', 'thrashers/none', 'thrashosds-health', 'workloads/rados_api_tests}'] 2021-12-08T01:10:19.819 INFO:scrape: 2021-12-08T01:10:19.820 INFO:scrape:Failure: Command failed on smithi094 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,nofallback' 2021-12-08T01:10:19.820 INFO:scrape:15 jobs: ['6549153', '6548940', '6548916', '6548988', '6549044', '6549097', '6549285', '6549130', '6549200', '6548879', '6549182', '6549235', '6548968', '6549076', '6549022'] 2021-12-08T01:10:19.821 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'syntax', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-12-08T01:10:19.821 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'syntax', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-12-08T01:10:19.822 INFO:scrape: 2021-12-08T01:10:19.822 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi063.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2021-12-08T01:10:19.823 INFO:scrape:2 jobs: ['6548981', '6549194'] 2021-12-08T01:10:19.823 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-12-08T01:10:19.824 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-12-08T01:10:19.824 INFO:scrape: 2021-12-08T01:10:19.824 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi115 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-12-08T01:10:19.825 INFO:scrape:2 jobs: ['6548980', '6549193'] 2021-12-08T01:10:19.825 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2021-12-08T01:10:19.826 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-lz4}', 'random-objectstore$/{bluestore-hybrid}', 'tasks/e2e}'] 2021-12-08T01:10:19.826 INFO:scrape: 2021-12-08T01:10:19.827 INFO:scrape:Failure: 2.0 not in backfilling 2021-12-08T01:10:19.827 INFO:scrape:1 jobs: ['6549036'] 2021-12-08T01:10:19.828 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/none', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'rados', 'rados/singleton/{all/backfill-toofull', 'supported-random-distro$/{centos_8.stream}}'] 2021-12-08T01:10:19.828 INFO:scrape: 2021-12-08T01:10:19.829 INFO:scrape:Failure: Command failed (workunit test rados/test_python.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh' 2021-12-08T01:10:19.829 INFO:scrape:1 jobs: ['6548902'] 2021-12-08T01:10:19.830 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_python}'] 2021-12-08T01:10:19.830 INFO:scrape: 2021-12-08T01:10:19.831 INFO:scrape:Failure: "2021-12-07T07:38:45.778996+0000 osd.4 (osd.4) 3 : cluster [WRN] Monitor daemon marked osd.4 down, but it is still running" in cluster log 2021-12-08T01:10:19.831 INFO:scrape:1 jobs: ['6548865'] 2021-12-08T01:10:19.831 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/none', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'rados', 'rados/singleton/{all/pg-autoscaler', 'supported-random-distro$/{centos_8}}'] 2021-12-08T01:10:19.832 INFO:scrape: 2021-12-08T01:10:19.832 INFO:scrape:Dead: b'2021-12-07T23:18:13.067 DEBUG:teuthology.exit:Finished running handlers' 2021-12-08T01:10:19.833 INFO:scrape:2 jobs: ['6549034', '6549244'] 2021-12-08T01:10:19.833 INFO:scrape:suites intersection: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2021-12-08T01:10:19.834 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2021-12-08T01:10:19.834 INFO:scrape: 2021-12-08T01:10:19.835 INFO:scrape:Failure: Command failed on smithi082 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 186b2692-5760-11ec-8c2e-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_4' 2021-12-08T01:10:19.835 INFO:scrape:1 jobs: ['6549263'] 2021-12-08T01:10:19.836 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'syntax', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-12-08T01:10:19.836 INFO:scrape: 2021-12-08T01:10:19.837 INFO:scrape:Failure: Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest) 2021-12-08T01:10:19.837 INFO:scrape:2 jobs: ['6549178', '6548964'] 2021-12-08T01:10:19.837 INFO:scrape:suites intersection: ['debug/mgr', 'rados/mgr/{clusters/{2-node-mgr}', 'supported-random-distro$/{centos_8}', 'tasks/module_selftest}'] 2021-12-08T01:10:19.838 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/mgr/{clusters/{2-node-mgr}', 'random-objectstore$/{bluestore-hybrid}', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'supported-random-distro$/{centos_8}', 'tasks/module_selftest}'] 2021-12-08T01:10:19.838 INFO:scrape: 2021-12-08T01:10:19.839 INFO:scrape:Failure: Command failed on smithi186 with status 1: 'sudo yum -y install cephfs-java' 2021-12-08T01:10:19.839 INFO:scrape:1 jobs: ['6549117'] 2021-12-08T01:10:19.840 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8.stream}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2021-12-08T01:10:19.840 INFO:scrape: 2021-12-08T01:10:19.841 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2021-12-08T01:10:19.841 INFO:scrape:1 jobs: ['6549228'] 2021-12-08T01:10:19.842 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_8}', 'tasks/rados_api_tests}'] 2021-12-08T01:10:19.842 INFO:scrape: 2021-12-08T01:10:19.857 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/ failed: 34 dead: 2 running: 0 waiting: 0 queued: 0 passed: 391 Fail ================================================================= [6548865] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548865/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548865/ "2021-12-07T07:38:45.778996+0000 osd.4 (osd.4) 3 : cluster [WRN] Monitor daemon marked osd.4 down, but it is still running" in cluster log [6548879] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548879/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548879/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1af41b5f7374979937152be5c8d28a3 Command failed on smithi105 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6548902] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} ----------------------------------------------------------------- time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548902/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b8e874817cde4562b36427fd9e8b08a9 Command failed (workunit test rados/test_python.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh' [6548916] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548916/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548916/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58a003fdd22e4fb6831f3be959ed09aa Command failed on smithi042 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6548940] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548940/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548940/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29d83656e94446bc92a821914a2cc48b Command failed on smithi111 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6548964] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/module_selftest} ----------------------------------------------------------------- time: 00:16:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548964/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548964/ sentry: https://sentry.ceph.com/organizations/ceph/?query=31cca30040324688a102e2f7523c7aa0 Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest) [6548968] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548968/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548968/ sentry: https://sentry.ceph.com/organizations/ceph/?query=128ae9f743e44cf7a7d0959b4aecc900 Command failed on smithi112 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6548980] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} ----------------------------------------------------------------- time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548980/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548980/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5449e4055c5d4a9a9d873411fde2d718 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi115 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6548981] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:05:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548981/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548981/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9f93d3f63fa46c19e45f054279b991f [Errno 2] Cannot find file on the remote 'ubuntu@smithi063.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6548988] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548988/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548988/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5d48c305b6842258230ae89f2d72822 Command failed on smithi087 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549022] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549022/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549022/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c4e5e2a7fd1c4ea2afe28ae8615acb83 Command failed on smithi042 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549036] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} ----------------------------------------------------------------- time: 01:09:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549036/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549036/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4e7bcd9b0874c91abd28d6a8fc66513 2.0 not in backfilling [6549044] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549044/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549044/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e6211e5e47ad412b9e98caf1fdd686a7 Command failed on smithi117 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549076] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549076/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549076/ sentry: https://sentry.ceph.com/organizations/ceph/?query=544b47da435d4531b60327575d4f72b5 Command failed on smithi005 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549086] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/1.7.0} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549086/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549086/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f4c3af0cd5b34cc18ce6cbdf902d43ee No module named 'tasks.radosbench' [6549097] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549097/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549097/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5cd8579bbfa46868e8bed1d21f5375b Command failed on smithi038 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549098] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549098/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549098/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40bb5d6d66d042839c15aad24440f160 No module named 'tasks.workunit' [6549100] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:09:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549100/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549100/ sentry: https://sentry.ceph.com/organizations/ceph/?query=849ad457e74f4c268393b09c3f304a2f No module named 'tasks.cram' [6549104] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:05:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549104/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549104/ sentry: https://sentry.ceph.com/organizations/ceph/?query=162b468e9d414dfe8cd18ab898058fd4 No module named 'tasks.ceph' [6549116] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549116/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549116/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a4c44e0e2aae46749d309cd795830811 No module named 'tasks.nvme_loop' [6549117] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:05:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549117/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549117/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a0791996b4734b33958e2de5d231814c Command failed on smithi186 with status 1: 'sudo yum -y install cephfs- java' [6549128] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} ----------------------------------------------------------------- time: 00:05:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549128/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549128/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c1001f74e7004adabcb8cb862cf1730b No module named 'tasks' [6549130] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549130/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549130/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f7e940493314efa85f64bfce9707acc Command failed on smithi098 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549133] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:03:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549133/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549133/ sentry: https://sentry.ceph.com/organizations/ceph/?query=68ec54dc66384b1d924508d00a0856cf No module named 'tasks' [6549153] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549153/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549153/ sentry: https://sentry.ceph.com/organizations/ceph/?query=71ec8cef623b419e86d71a1d7f3ce401 Command failed on smithi094 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549178] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_8} tasks/module_selftest} ----------------------------------------------------------------- time: 00:16:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549178/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549178/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48cacd7942a84e8ba4def2c9f8bfab42 Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest) [6549182] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549182/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549182/ sentry: https://sentry.ceph.com/organizations/ceph/?query=51b3b85d326747df932fdc0e9a8fbdd8 Command failed on smithi007 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549193] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/e2e} ----------------------------------------------------------------- time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549193/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549193/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8faa1e4a695049ec92c1aac10a15c7be Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6549194] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549194/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549194/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed1f760a7efd45aaab12a3cc859aa61b [Errno 2] Cannot find file on the remote 'ubuntu@smithi159.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6549200] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549200/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549200/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cb47a247092e44e3a353b9c032cd56d4 Command failed on smithi098 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549228] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_api_tests} ----------------------------------------------------------------- time: 03:26:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549228/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549228/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2a01990e1f654b21a24b67a1edd6242a Command failed (workunit test rados/test.sh) on smithi038 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=92404026b1bda5d5ba9b717488ee32693a78ca83 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [6549235] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549235/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549235/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f7fa5c6c9c9841dc97105bd17f0527e8 Command failed on smithi039 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' [6549263] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549263/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549263/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be0d90aee7a74d3f8df2cf4ae29d40a0 Command failed on smithi082 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 186b2692-5760-11ec-8c2e-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_4' [6549285] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549285/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549285/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d90d0ba5b8fb4fc4bf4cd6071741b892 Command failed on smithi002 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs,nofallback' Dead ================================================================= [6549034] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549034/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549034/ hit max job timeout [6549244] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549244/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549244/ hit max job timeout Pass ================================================================= [6548859] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548859/ [6548860] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548860/ [6548861] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:42:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548861/ [6548862] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548862/ [6548863] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548863/ [6548864] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:33:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548864/ [6548866] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:20:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548866/ [6548867] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:36:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548867/ [6548868] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:15:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548868/ [6548869] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:48:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548869/ [6548870] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/dashboard} time: 01:11:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548870/ [6548871] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:09:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548871/ [6548872] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:20:26 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548872/ [6548873] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548873/ [6548874] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.0} time: 00:31:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548874/ [6548875] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:29:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548875/ [6548876] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/c2c} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548876/ [6548877] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:14:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548877/ [6548878] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:26:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548878/ [6548880] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:29:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548880/ [6548881] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548881/ [6548882] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:25:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548882/ [6548883] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:06:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548883/ [6548884] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:34:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548884/ [6548885] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548885/ [6548886] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548886/ [6548887] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548887/ [6548888] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548888/ [6548889] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548889/ [6548890] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548890/ [6548891] rados/cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548891/ [6548892] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:16:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548892/ [6548893] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:21:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548893/ [6548894] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548894/ [6548895] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.2_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:12:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548895/ [6548896] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:53 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548896/ [6548897] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:22:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548897/ [6548898] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548898/ [6548899] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:04 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548899/ [6548900] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:17:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548900/ [6548901] rados/cephadm/workunits/{agent/off mon_election/classic task/test_adoption} time: 00:07:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548901/ [6548903] rados/singleton/{all/random-eio mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream}} time: 00:21:21 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548903/ [6548904] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:39:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548904/ [6548905] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:21:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548905/ [6548906] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{rhel_8} tasks/failover} time: 00:26:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548906/ [6548907] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548907/ [6548908] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:24:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548908/ [6548909] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:07:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548909/ [6548910] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:21:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548910/ [6548911] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548911/ [6548912] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:10:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548912/ [6548913] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:20:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548913/ [6548914] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:28:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548914/ [6548915] rados/cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548915/ [6548917] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:10:13 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548917/ [6548918] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:09:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548918/ [6548919] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits} time: 00:30:26 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548919/ [6548920] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548920/ [6548921] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:31:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548921/ [6548922] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects} time: 00:21:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548922/ [6548923] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:27:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548923/ [6548924] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 01:33:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548924/ [6548925] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:44:49 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548925/ [6548926] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548926/ [6548927] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:50:06 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548927/ [6548928] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548928/ [6548929] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548929/ [6548930] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548930/ [6548931] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:10:51 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548931/ [6548932] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548932/ [6548933] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:17 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548933/ [6548934] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548934/ [6548935] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:16:18 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548935/ [6548936] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548936/ [6548937] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548937/ [6548938] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:34:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548938/ [6548939] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:39:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548939/ [6548941] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:31:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548941/ [6548942] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548942/ [6548943] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548943/ [6548944] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:49:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548944/ [6548945] rados/singleton/{all/test-crash mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream}} time: 00:09:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548945/ [6548946] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548946/ [6548947] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:44:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548947/ [6548948] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:19:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548948/ [6548949] rados/cephadm/workunits/{agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548949/ [6548950] rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} time: 00:06:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548950/ [6548951] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:09:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548951/ [6548952] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:29:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548952/ [6548953] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:09:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548953/ [6548954] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548954/ [6548955] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548955/ [6548956] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:20:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548956/ [6548957] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548957/ [6548958] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:22:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548958/ [6548959] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548959/ [6548960] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:17:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548960/ [6548961] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/one workloads/snaps-few-objects} time: 00:33:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548961/ [6548962] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:19:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548962/ [6548963] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548963/ [6548965] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548965/ [6548966] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:07:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548966/ [6548967] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:42:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548967/ [6548969] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548969/ [6548970] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:39:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548970/ [6548971] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 01:29:01 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548971/ [6548972] rados/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:40:11 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548972/ [6548973] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:17 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548973/ [6548974] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548974/ [6548975] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:48:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548975/ [6548976] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548976/ [6548977] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548977/ [6548978] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:16:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548978/ [6548979] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:29:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548979/ [6548982] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548982/ [6548983] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 01:08:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548983/ [6548984] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench} time: 01:26:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548984/ [6548985] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_nfs} time: 00:28:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548985/ [6548986] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:17 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548986/ [6548987] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:21:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548987/ [6548989] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548989/ [6548990] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{ubuntu_latest}} time: 02:46:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548990/ [6548991] rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/on orchestrator_cli} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548991/ [6548992] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:33:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548992/ [6548993] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:08:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548993/ [6548994] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548994/ [6548995] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:19:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548995/ [6548996] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548996/ [6548997] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548997/ [6548998] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:24:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548998/ [6548999] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:50:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6548999/ [6549000] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} tasks/rados_workunit_loadgen_big} time: 00:27:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549000/ [6549001] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:20:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549001/ [6549002] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:31:49 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549002/ [6549003] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:10:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549003/ [6549004] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549004/ [6549005] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:43:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549005/ [6549006] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549006/ [6549007] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549007/ [6549008] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549008/ [6549009] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549009/ [6549010] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:39:17 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549010/ [6549011] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549011/ [6549012] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:09:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549012/ [6549013] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:19:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549013/ [6549014] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549014/ [6549015] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:16:01 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549015/ [6549016] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} time: 00:26:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549016/ [6549017] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:26:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549017/ [6549018] rados/cephadm/workunits/{agent/off mon_election/classic task/test_orch_cli} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549018/ [6549019] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:33:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549019/ [6549020] rados/cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:31:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549020/ [6549021] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:20:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549021/ [6549023] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:19:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549023/ [6549024] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549024/ [6549025] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549025/ [6549026] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:25:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549026/ [6549027] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:34:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549027/ [6549028] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549028/ [6549029] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:42:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549029/ [6549030] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549030/ [6549031] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549031/ [6549032] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549032/ [6549033] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} tasks/rados_workunit_loadgen_mix} time: 00:18:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549033/ [6549035] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549035/ [6549037] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:13:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549037/ [6549038] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:47:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549038/ [6549039] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549039/ [6549040] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:06:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549040/ [6549041] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:35:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549041/ [6549042] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:32:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549042/ [6549043] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:22:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549043/ [6549045] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream}} time: 00:08:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549045/ [6549046] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:25:10 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549046/ [6549047] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549047/ [6549048] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549048/ [6549049] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:22:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549049/ [6549050] rados/cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549050/ [6549051] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:51 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549051/ [6549052] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549052/ [6549053] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:23:06 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549053/ [6549054] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_adoption} time: 00:07:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549054/ [6549055] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{centos_8} tasks/workunits} time: 00:10:11 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549055/ [6549056] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549056/ [6549057] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:11:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549057/ [6549058] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:20:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549058/ [6549059] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:35:05 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549059/ [6549060] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:39:57 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549060/ [6549061] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:54:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549061/ [6549062] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/dedup-io-mixed} time: 00:09:53 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549062/ [6549063] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:31:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549063/ [6549064] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549064/ [6549065] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549065/ [6549066] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:36:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549066/ [6549067] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:08:38 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549067/ [6549068] rados/cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549068/ [6549069] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream}} time: 00:08:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549069/ [6549070] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:32:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549070/ [6549071] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549071/ [6549072] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:32:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549072/ [6549073] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549073/ [6549074] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549074/ [6549075] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549075/ [6549077] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:06 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549077/ [6549078] rados/cephadm/workunits/{agent/off mon_election/classic task/test_cephadm} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549078/ [6549079] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:22:01 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549079/ [6549080] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549080/ [6549081] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:19:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549081/ [6549082] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:09:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549082/ [6549083] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:39:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549083/ [6549084] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:12:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549084/ [6549085] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{filestore-xfs} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:09:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549085/ [6549087] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:31:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549087/ [6549088] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:14:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549088/ [6549089] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:10:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549089/ [6549090] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:42:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549090/ [6549091] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549091/ [6549092] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:37:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549092/ [6549093] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:39:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549093/ [6549094] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 01:08:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549094/ [6549095] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 01:06:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549095/ [6549096] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549096/ [6549099] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:13:01 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549099/ [6549101] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549101/ [6549102] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549102/ [6549103] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/off orchestrator_cli} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549103/ [6549105] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:32:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549105/ [6549106] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549106/ [6549107] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:25:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549107/ [6549108] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:47:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549108/ [6549109] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549109/ [6549110] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:23:30 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549110/ [6549111] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:32:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549111/ [6549112] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:24:18 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549112/ [6549113] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:04:51 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549113/ [6549114] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:04:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549114/ [6549115] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:06:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549115/ [6549118] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/failover} time: 00:15:00 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549118/ [6549119] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:46:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549119/ [6549120] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:10:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549120/ [6549121] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549121/ [6549122] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549122/ [6549123] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549123/ [6549124] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} tasks/mon_recovery} time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549124/ [6549125] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:45:02 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549125/ [6549126] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 01:00:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549126/ [6549127] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549127/ [6549129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/redirect} time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549129/ [6549131] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549131/ [6549132] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:12:04 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549132/ [6549134] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:51:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549134/ [6549135] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549135/ [6549136] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:26:10 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549136/ [6549137] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549137/ [6549138] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549138/ [6549139] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549139/ [6549140] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:36:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549140/ [6549141] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549141/ [6549142] rados/cephadm/workunits/{agent/off mon_election/classic task/test_nfs} time: 00:24:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549142/ [6549143] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:10:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549143/ [6549144] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:29:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549144/ [6549145] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:56 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549145/ [6549146] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:19:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549146/ [6549147] rados/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:43:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549147/ [6549148] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{filestore-xfs} supported-random-distro$/{centos_8.stream} tasks/insights} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549148/ [6549149] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:36:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549149/ [6549150] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:20:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549150/ [6549151] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549151/ [6549152] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549152/ [6549154] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:31:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549154/ [6549155] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549155/ [6549156] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd-backfill} time: 04:20:27 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549156/ [6549157] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:25:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549157/ [6549158] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:11:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549158/ [6549159] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549159/ [6549160] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549160/ [6549161] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:11 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549161/ [6549162] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:15:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549162/ [6549163] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:16:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549163/ [6549164] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549164/ [6549165] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549165/ [6549166] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} tasks/mon_clock_no_skews} time: 00:06:49 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549166/ [6549167] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:23:35 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549167/ [6549168] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:04 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549168/ [6549169] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:07:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549169/ [6549170] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:36:21 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549170/ [6549171] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549171/ [6549172] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream}} time: 00:30:21 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549172/ [6549173] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:20:10 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549173/ [6549174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:19:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549174/ [6549175] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 01:04:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549175/ [6549176] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:27:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549176/ [6549177] rados/cephadm/workunits/{agent/on mon_election/connectivity task/test_orch_cli} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549177/ [6549179] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:21:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549179/ [6549180] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:08:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549180/ [6549181] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549181/ [6549183] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549183/ [6549184] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} time: 02:24:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549184/ [6549185] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 01:12:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549185/ [6549186] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:34:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549186/ [6549187] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549187/ [6549188] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:50 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549188/ [6549189] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549189/ [6549190] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:24:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549190/ [6549191] rados/cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:48:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549191/ [6549192] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:31:41 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549192/ [6549195] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:07:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549195/ [6549196] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549196/ [6549197] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:22:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549197/ [6549198] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:30:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549198/ [6549199] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:22:55 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549199/ [6549201] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:37:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549201/ [6549202] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549202/ [6549203] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549203/ [6549204] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549204/ [6549205] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:11 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549205/ [6549206] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 03:23:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549206/ [6549207] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:28:16 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549207/ [6549208] rados/cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:16:11 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549208/ [6549209] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/progress} time: 00:35:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549209/ [6549210] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:10:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549210/ [6549211] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:40 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549211/ [6549212] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549212/ [6549213] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549213/ [6549214] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:07:38 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549214/ [6549215] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549215/ [6549216] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:18:36 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549216/ [6549217] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:22:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549217/ [6549218] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:24:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549218/ [6549219] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:39:18 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549219/ [6549220] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:27:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549220/ [6549221] rados/cephadm/workunits/{agent/on mon_election/classic task/test_adoption} time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549221/ [6549222] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549222/ [6549223] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8.stream}} time: 02:15:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549223/ [6549224] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549224/ [6549225] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:30:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549225/ [6549226] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:21:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549226/ [6549227] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549227/ [6549229] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:35:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549229/ [6549230] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:23:23 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549230/ [6549231] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549231/ [6549232] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549232/ [6549233] rados/cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549233/ [6549234] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:37:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549234/ [6549236] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:21:25 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549236/ [6549237] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549237/ [6549238] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549238/ [6549239] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{centos_8.stream} tasks/prometheus} time: 00:11:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549239/ [6549240] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549240/ [6549241] rados/cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549241/ [6549242] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549242/ [6549243] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:29 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549243/ [6549245] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:32:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549245/ [6549246] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:11:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549246/ [6549247] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:39 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549247/ [6549248] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/repave-all} time: 00:09:22 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549248/ [6549249] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549249/ [6549250] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:29:56 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549250/ [6549251] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 02:59:37 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549251/ [6549252] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549252/ [6549253] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549253/ [6549254] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:09:17 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549254/ [6549255] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:58:32 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549255/ [6549256] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:44:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549256/ [6549257] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549257/ [6549258] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:47:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549258/ [6549259] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:18:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549259/ [6549260] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/sync workloads/pool-create-delete} time: 00:13:31 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549260/ [6549261] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-big} time: 00:27:08 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549261/ [6549262] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{centos_8.stream}} time: 00:28:09 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549262/ [6549264] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:31:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549264/ [6549265] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:07 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549265/ [6549266] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:18:14 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549266/ [6549267] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:48 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549267/ [6549268] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8.stream} tasks/workunits} time: 00:08:15 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549268/ [6549269] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:22:43 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549269/ [6549270] rados/cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos} time: 00:04:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549270/ [6549271] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:51 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549271/ [6549272] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549272/ [6549273] rados/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:47:46 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549273/ [6549274] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:33:03 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549274/ [6549275] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:09:47 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549275/ [6549276] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:28:52 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549276/ [6549277] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:37:59 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549277/ [6549278] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:10:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549278/ [6549279] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549279/ [6549280] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:58 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549280/ [6549281] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:21:12 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549281/ [6549282] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:18:34 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549282/ [6549283] rados/thrash-old-clients/{0-distro$/{centos_8.3_container_tools_3.0} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:47:44 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549283/ [6549284] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:42 info: http://pulpito.ceph.com/yuriw-2021-12-07_00:28:11-rados-wip-master_12.6.21-distro-default-smithi/6549284/