2022-04-27T02:55:33.367 INFO:root:teuthology version: 1.1.1.dev326+g135488ac 2022-04-27T02:55:33.367 INFO:teuthology.results:subset: '111/120000' 2022-04-27T02:55:33.368 INFO:teuthology.results:seed: '230' 2022-04-27T07:07:15.647 INFO:root:teuthology version: 1.1.1.dev326+g135488ac 2022-04-27T07:07:15.687 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-27T13:29:53.670 INFO:teuthology.results:Tests finished! gathering results... 2022-04-27T13:30:26.226 INFO:scrape:Found 420 jobs 2022-04-27T13:30:29.181 INFO:scrape:Found 9 distinct failure reasons 2022-04-27T13:30:29.182 INFO:scrape:Crash: Command failed (workunit test cls/test_cls_rbd.sh) on smithi196 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=nautilus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' 2022-04-27T13:30:29.182 INFO:scrape:ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable) 1: (()+0x12980) [0x7fad0f900980] 2: (gsignal()+0xc7) [0x7fad0e9f8e87] 3: (abort()+0x141) [0x7fad0e9fa7f1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x7fad10a87483] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7fad10a8760d] 6: (FSMap::sanity() const+0xe0) [0x7fad10fabb80] 7: (MDSMonitor::update_from_paxos(bool*)+0x488) [0x5585395487c8] 8: (PaxosService::refresh(bool*)+0x25a) [0x55853948377a] 9: (Monitor::refresh_from_paxos(bool*)+0x10c) [0x55853936032c] 10: (Paxos::do_refresh()+0x4f) [0x5585394727cf] 11: (Paxos::handle_commit(boost::intrusive_ptr)+0x153) [0x558539477f03] 12: (Paxos::dispatch(boost::intrusive_ptr)+0x2f3) [0x55853947dcf3] 13: (Monitor::dispatch_op(boost::intrusive_ptr)+0x19ef) [0x558539394a6f] 14: (Monitor::_ms_dispatch(Message*)+0x50a) [0x558539394fea] 15: (Monitor::ms_dispatch(Message*)+0x26) [0x5585393c6586] 16: (Dispatcher::ms_dispatch2(boost::intrusive_ptr const&)+0x26) [0x5585393c2636] 17: (DispatchQueue::entry()+0x1219) [0x7fad10cb7bc9] 18: (DispatchQueue::DispatchThread::entry()+0xd) [0x7fad10d68e3d] 19: (()+0x76db) [0x7fad0f8f56db] 20: (clone()+0x3f) [0x7fad0eadb61f] 2022-04-27T13:30:29.182 INFO:scrape:2 jobs: ['6807946', '6807739'] 2022-04-27T13:30:29.183 INFO:scrape:suites intersection: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-pacific', '8-workload/{rbd-python', 'bluestore-bitmap', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}'] 2022-04-27T13:30:29.183 INFO:scrape:suites union: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-pacific', '8-workload/{rbd-python', 'bluestore-bitmap', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}'] 2022-04-27T13:30:29.183 INFO:scrape: 2022-04-27T13:30:29.183 INFO:scrape:Dead: b'2022-04-27T13:04:17.791 DEBUG:teuthology.exit:Finished running handlers' 2022-04-27T13:30:29.183 INFO:scrape:7 jobs: ['6808044', '6807726', '6807889', '6808078', '6807840', '6808006', '6807954'] 2022-04-27T13:30:29.183 INFO:scrape:suites intersection: [] 2022-04-27T13:30:29.184 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/octopus', '1-ranks/1', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-nfs', '2-ops/rm-zap-flag}', '3-inline/no', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'cluster/{12-osds', 'conf/{client', 'distro/rhel_8.4_container_tools_rhel8', 'distro/ubuntu_20.04', 'fixed-2', 'mds', 'mon', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-stupid', 'openstack}', 'osd}', 'overrides/{pg-warn', 'rados', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-active-recovery}', 'roles', 'start}', 'supported-random-distro$/{rhel_8}', 'tasks/{0-from/v16.2.4', 'thrashers/pggrow', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2022-04-27T13:30:29.184 INFO:scrape: 2022-04-27T13:30:29.184 INFO:scrape:Failure: "2022-04-27T05:08:22.852563+0000 osd.6 (osd.6) 3 : cluster [ERR] osd.6 found snap mapper error on pg 3.8s2 oid 3:1758620f:::smithi06683418-27:aa snaps missing in mapper, should be: a8,a9 was r -2...repaired" in cluster log 2022-04-27T13:30:29.184 INFO:scrape:2 jobs: ['6807891', '6807802'] 2022-04-27T13:30:29.184 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'openstack}', 'rados', 'thrashers/default', 'thrashosds-health'] 2022-04-27T13:30:29.184 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}'] 2022-04-27T13:30:29.185 INFO:scrape: 2022-04-27T13:30:29.185 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi061.front.sepia.ceph.com: ['type=AVC msg=audit(1651031923.879:17922): avc: denied { ioctl } for pid=107035 comm="iptables" path="/var/lib/containers/storage/overlay/55dc31c6cf2aed294987bac42e473f6c618f13ca394b87db3e3876d51cd2feb0/merged" dev="overlay" ino=3412412 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2022-04-27T13:30:29.185 INFO:scrape:2 jobs: ['6807779', '6808089'] 2022-04-27T13:30:29.185 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-27T13:30:29.185 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-04-27T13:30:29.185 INFO:scrape: 2022-04-27T13:30:29.185 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2022-04-27T13:30:29.185 INFO:scrape:2 jobs: ['6807870', '6807874'] 2022-04-27T13:30:29.186 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start'] 2022-04-27T13:30:29.186 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', '2-services/iscsi', '3-final}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04'] 2022-04-27T13:30:29.186 INFO:scrape: 2022-04-27T13:30:29.186 INFO:scrape:Failure: failed to complete snap trimming before timeout 2022-04-27T13:30:29.186 INFO:scrape:1 jobs: ['6807707'] 2022-04-27T13:30:29.186 INFO:scrape:suites: ['cluster/{12-osds', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-async-partial-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2022-04-27T13:30:29.186 INFO:scrape: 2022-04-27T13:30:29.187 INFO:scrape:Failure: Command failed on smithi033 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc706450-c5dd-11ec-8c39-001a4aab830c -- ceph orch daemon add osd smithi033:vg_nvme/lv_4' 2022-04-27T13:30:29.187 INFO:scrape:6 jobs: ['6807786', '6807910', '6808015', '6807746', '6808096', '6807957'] 2022-04-27T13:30:29.187 INFO:scrape:suites intersection: ['3-upgrade-with-workload'] 2022-04-27T13:30:29.187 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-nfs', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'roles', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-27T13:30:29.187 INFO:scrape: 2022-04-27T13:30:29.187 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi154.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2022-04-27T13:30:29.188 INFO:scrape:2 jobs: ['6807710', '6807974'] 2022-04-27T13:30:29.188 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '3-final', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-04-27T13:30:29.188 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-04-27T13:30:29.188 INFO:scrape: 2022-04-27T13:30:29.188 INFO:scrape:Failure: Command failed (workunit test rados/test_python.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4fa079ba14503defa8dc257d7c2d506ebefcfe6d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh' 2022-04-27T13:30:29.188 INFO:scrape:1 jobs: ['6807766'] 2022-04-27T13:30:29.188 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_python}'] 2022-04-27T13:30:29.189 INFO:scrape: 2022-04-27T13:30:29.201 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/ failed: 18 dead: 7 running: 0 waiting: 0 queued: 0 passed: 395 Fail ================================================================= [6807707] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 01:26:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807707/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807707/ failed to complete snap trimming before timeout [6807710] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:44 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807710/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807710/ sentry: https://sentry.ceph.com/organizations/ceph/?query=833788254aac409e87847d594f2d3031 [Errno 2] Cannot find file on the remote 'ubuntu@smithi154.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6807739] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807739/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807739/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a6cb5f5bc5e741db97f108989232d88d Command failed (workunit test cls/test_cls_rbd.sh) on smithi146 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=nautilus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [6807746] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:20:56 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807746/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807746/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8bfc825f34674c7d87b3e1cd65e991bb Command failed on smithi115 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a24c12e-c5da-11ec-8c39-001a4aab830c -e sha1=4fa079ba14503defa8dc257d7c2d506ebefcfe6d -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6807766] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} ----------------------------------------------------------------- time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807766/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807766/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1044ac475ab741f9b2552e05cf479a42 Command failed (workunit test rados/test_python.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4fa079ba14503defa8dc257d7c2d506ebefcfe6d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh' [6807779] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:15:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807779/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807779/ SELinux denials found on ubuntu@smithi061.front.sepia.ceph.com: ['type=AVC msg=audit(1651031923.879:17922): avc: denied { ioctl } for pid=107035 comm="iptables" path="/var/lib/containers/storage/overlay/55dc31c6cf2aed294 987bac42e473f6c618f13ca394b87db3e3876d51cd2feb0/merged" dev="overlay" ino=3412412 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6807786] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807786/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807786/ sentry: https://sentry.ceph.com/organizations/ceph/?query=56243b270b3e423e90bd75456e10b372 Command failed on smithi033 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc706450-c5dd-11ec-8c39-001a4aab830c -- ceph orch daemon add osd smithi033:vg_nvme/lv_4' [6807802] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} ----------------------------------------------------------------- time: 00:27:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807802/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807802/ "2022-04-27T04:18:33.115282+0000 osd.0 (osd.0) 138 : cluster [ERR] 3.35s2 shard 0(2) 3:affd9c95:::smithi05232360-26:25e : missing" in cluster log [6807870] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} ----------------------------------------------------------------- time: 00:22:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807870/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807870/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7b124be01b7416f80b962bf223b6192 reached maximum tries (180) after waiting for 180 seconds [6807874] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:22:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807874/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=989462cd589840b4a33cc41061c0513f reached maximum tries (180) after waiting for 180 seconds [6807891] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:44:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807891/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807891/ "2022-04-27T05:08:22.852563+0000 osd.6 (osd.6) 3 : cluster [ERR] osd.6 found snap mapper error on pg 3.8s2 oid 3:1758620f:::smithi06683418-27:aa snaps missing in mapper, should be: a8,a9 was r -2...repaired" in cluster log [6807910] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:51:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807910/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=066c66050a6c4013940bfbf1eedbb95e Command failed on smithi124 with status 110: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 594367f2-c5e6-11ec-8c39-001a4aab830c -e sha1=4fa079ba14503defa8dc257d7c2d506ebefcfe6d -- bash -c 'ceph orch ps'" [6807946] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:17:01 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807946/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807946/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c2b2694390484f288499034b35b63419 Command failed (workunit test cls/test_cls_rbd.sh) on smithi196 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=nautilus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [6807957] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:20:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807957/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807957/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f82e4d40b48b4e53bc5949cf5d753df5 Command failed on smithi135 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9cd96e8c-c5e9-11ec-8c39-001a4aab830c -e sha1=4fa079ba14503defa8dc257d7c2d506ebefcfe6d -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [6807974] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:07:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807974/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807974/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c3afa6c768b4ac79b96376b68274dae [Errno 2] Cannot find file on the remote 'ubuntu@smithi107.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' [6808015] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808015/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808015/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c0e50c843a94ddfad3134f7b1d5b9b7 Command failed on smithi145 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb99b7a6-c5f1-11ec-8c39-001a4aab830c -- ceph orch daemon add osd smithi145:vg_nvme/lv_4' [6808089] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:15:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808089/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808089/ SELinux denials found on ubuntu@smithi188.front.sepia.ceph.com: ['type=AVC msg=audit(1651043889.818:17931): avc: denied { ioctl } for pid=107175 comm="iptables" path="/var/lib/containers/storage/overlay/77e4459000f1d2e18 ac3eace90eb39b39d4afba8f022f9942561a6554ea40eb9/merged" dev="overlay" ino=3412407 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [6808096] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:21:32 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808096/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808096/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e1750d6ec314f86a8123820f5717baf Command failed on smithi120 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6df9c0b6-c5f9-11ec-8c39-001a4aab830c -e sha1=4fa079ba14503defa8dc257d7c2d506ebefcfe6d -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' Dead ================================================================= [6807726] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807726/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807726/ hit max job timeout [6807840] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807840/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807840/ hit max job timeout [6807889] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807889/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807889/ hit max job timeout [6807954] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807954/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807954/ hit max job timeout [6808006] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808006/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808006/ hit max job timeout [6808044] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808044/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808044/ hit max job timeout [6808078] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808078/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808078/ hit max job timeout Pass ================================================================= [6807683] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:44:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807683/ [6807684] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:25:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807684/ [6807685] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807685/ [6807686] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:17:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807686/ [6807687] rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} time: 04:45:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807687/ [6807688] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807688/ [6807689] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:06:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807689/ [6807690] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807690/ [6807691] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:35:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807691/ [6807692] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:18:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807692/ [6807693] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} time: 01:26:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807693/ [6807694] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807694/ [6807695] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807695/ [6807696] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:52:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807696/ [6807697] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:30:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807697/ [6807698] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:25:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807698/ [6807699] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/prometheus} time: 00:12:56 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807699/ [6807700] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:38:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807700/ [6807701] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:24:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807701/ [6807702] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807702/ [6807703] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807703/ [6807704] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807704/ [6807705] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:29:36 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807705/ [6807706] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807706/ [6807708] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:28:57 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807708/ [6807709] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807709/ [6807711] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:43:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807711/ [6807712] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807712/ [6807713] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:25:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807713/ [6807714] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:39:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807714/ [6807715] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:42:22 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807715/ [6807716] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:37:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807716/ [6807717] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:15:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807717/ [6807718] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:17:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807718/ [6807719] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807719/ [6807720] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807720/ [6807721] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807721/ [6807722] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807722/ [6807723] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807723/ [6807724] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:46:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807724/ [6807725] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:30:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807725/ [6807727] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807727/ [6807728] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807728/ [6807729] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:18:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807729/ [6807730] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807730/ [6807731] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:34:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807731/ [6807732] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:09:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807732/ [6807733] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:18:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807733/ [6807734] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:13:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807734/ [6807735] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:19:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807735/ [6807736] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807736/ [6807737] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807737/ [6807738] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:23:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807738/ [6807740] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:23:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807740/ [6807741] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:23:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807741/ [6807742] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:01 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807742/ [6807743] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807743/ [6807744] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/workunits} time: 00:19:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807744/ [6807745] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:24:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807745/ [6807747] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:26:06 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807747/ [6807748] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:22:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807748/ [6807749] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:23:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807749/ [6807750] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807750/ [6807751] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:31:07 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807751/ [6807752] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807752/ [6807753] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807753/ [6807754] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:07:44 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807754/ [6807755] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807755/ [6807756] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807756/ [6807757] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:23:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807757/ [6807758] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:11:37 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807758/ [6807759] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:24:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807759/ [6807760] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:35:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807760/ [6807761] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:33:01 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807761/ [6807762] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:56 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807762/ [6807763] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:25:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807763/ [6807764] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807764/ [6807765] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:45:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807765/ [6807767] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:10:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807767/ [6807768] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807768/ [6807769] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:25:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807769/ [6807770] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:24:57 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807770/ [6807771] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807771/ [6807772] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:21:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807772/ [6807773] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807773/ [6807774] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:17 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807774/ [6807775] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:09:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807775/ [6807776] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807776/ [6807777] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 01:05:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807777/ [6807778] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:31:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807778/ [6807780] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:07:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807780/ [6807781] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:34:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807781/ [6807782] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/crash} time: 00:21:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807782/ [6807783] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits} time: 00:30:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807783/ [6807784] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:23:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807784/ [6807785] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:50:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807785/ [6807787] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807787/ [6807788] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807788/ [6807789] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:16:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807789/ [6807790] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:10:37 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807790/ [6807791] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:45:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807791/ [6807792] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807792/ [6807793] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:26:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807793/ [6807794] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807794/ [6807795] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807795/ [6807796] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807796/ [6807797] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807797/ [6807798] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:42:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807798/ [6807799] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:42 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807799/ [6807800] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:06:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807800/ [6807801] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807801/ [6807803] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:32:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807803/ [6807804] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:23:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807804/ [6807805] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807805/ [6807806] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807806/ [6807807] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:02 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807807/ [6807808] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:38:44 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807808/ [6807809] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:30:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807809/ [6807810] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:09:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807810/ [6807811] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807811/ [6807812] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:21:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807812/ [6807813] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:27:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807813/ [6807814] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807814/ [6807815] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807815/ [6807816] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:25:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807816/ [6807817] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:32:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807817/ [6807818] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:29:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807818/ [6807819] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807819/ [6807820] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:27:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807820/ [6807821] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807821/ [6807822] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:14:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807822/ [6807823] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807823/ [6807824] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/failover} time: 00:23:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807824/ [6807825] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/one workloads/snaps-few-objects} time: 00:47:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807825/ [6807826] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807826/ [6807827] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807827/ [6807828] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:32:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807828/ [6807829] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:08:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807829/ [6807830] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:29:56 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807830/ [6807831] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:39:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807831/ [6807832] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807832/ [6807833] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 01:22:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807833/ [6807834] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807834/ [6807835] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:40 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807835/ [6807836] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807836/ [6807837] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807837/ [6807838] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:40:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807838/ [6807839] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:07:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807839/ [6807841] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:16:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807841/ [6807842] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} time: 00:29:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807842/ [6807843] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 01:14:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807843/ [6807844] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:34:06 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807844/ [6807845] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807845/ [6807846] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:43:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807846/ [6807847] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:16:01 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807847/ [6807848] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807848/ [6807849] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:47:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807849/ [6807850] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807850/ [6807851] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807851/ [6807852] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807852/ [6807853] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:21:44 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807853/ [6807854] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:13:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807854/ [6807855] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:10:42 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807855/ [6807856] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:37:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807856/ [6807857] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807857/ [6807858] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807858/ [6807859] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:29:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807859/ [6807860] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807860/ [6807861] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:14:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807861/ [6807862] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807862/ [6807863] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:27:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807863/ [6807864] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:10:42 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807864/ [6807865] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:31:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807865/ [6807866] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807866/ [6807867] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/insights} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807867/ [6807868] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:26:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807868/ [6807869] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807869/ [6807871] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:27:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807871/ [6807872] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807872/ [6807873] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:30:19 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807873/ [6807875] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:19:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807875/ [6807876] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807876/ [6807877] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:19:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807877/ [6807878] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:26:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807878/ [6807879] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:22:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807879/ [6807880] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:41:46 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807880/ [6807881] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807881/ [6807882] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807882/ [6807883] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807883/ [6807884] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:31:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807884/ [6807885] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:32:29 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807885/ [6807886] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:21:36 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807886/ [6807887] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:07:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807887/ [6807888] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:31:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807888/ [6807890] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:14:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807890/ [6807892] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} time: 00:20:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807892/ [6807893] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:33:44 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807893/ [6807894] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807894/ [6807895] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:36:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807895/ [6807896] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807896/ [6807897] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:47:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807897/ [6807898] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807898/ [6807899] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:25:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807899/ [6807900] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:07:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807900/ [6807901] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:06:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807901/ [6807902] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:46 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807902/ [6807903] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:19:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807903/ [6807904] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:09:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807904/ [6807905] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:24:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807905/ [6807906] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:32:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807906/ [6807907] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:35:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807907/ [6807908] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807908/ [6807909] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:09:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807909/ [6807911] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:40 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807911/ [6807912] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:25:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807912/ [6807913] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:59:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807913/ [6807914] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807914/ [6807915] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:30:54 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807915/ [6807916] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807916/ [6807917] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:19:57 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807917/ [6807918] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:22 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807918/ [6807919] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:12:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807919/ [6807920] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:25:32 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807920/ [6807921] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:44:22 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807921/ [6807922] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} time: 02:34:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807922/ [6807923] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:51:53 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807923/ [6807924] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:31:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807924/ [6807925] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:09:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807925/ [6807926] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:24:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807926/ [6807927] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:34:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807927/ [6807928] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807928/ [6807929] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:20:56 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807929/ [6807930] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:42:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807930/ [6807931] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807931/ [6807932] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807932/ [6807933] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:40:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807933/ [6807934] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807934/ [6807935] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807935/ [6807936] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807936/ [6807937] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:30:57 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807937/ [6807938] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:09:32 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807938/ [6807939] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807939/ [6807940] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807940/ [6807941] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:09:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807941/ [6807942] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:41:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807942/ [6807943] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807943/ [6807944] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_8} tasks/dashboard} time: 01:00:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807944/ [6807945] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:28:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807945/ [6807947] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:45:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807947/ [6807948] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 01:22:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807948/ [6807949] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:08:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807949/ [6807950] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807950/ [6807951] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:21:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807951/ [6807952] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/progress} time: 00:17:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807952/ [6807953] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:27:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807953/ [6807955] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807955/ [6807956] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:47:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807956/ [6807958] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807958/ [6807959] rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr orchestrator_cli} time: 00:10:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807959/ [6807960] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:41:39 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807960/ [6807961] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807961/ [6807962] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:22:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807962/ [6807963] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 01:05:40 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807963/ [6807964] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:14:29 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807964/ [6807965] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807965/ [6807966] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/radosbench} time: 00:57:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807966/ [6807967] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:14:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807967/ [6807968] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:27:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807968/ [6807969] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:18:19 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807969/ [6807970] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:31:17 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807970/ [6807971] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:06:46 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807971/ [6807972] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807972/ [6807973] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807973/ [6807975] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:37:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807975/ [6807976] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:02 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807976/ [6807977] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:42:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807977/ [6807978] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:10:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807978/ [6807979] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807979/ [6807980] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807980/ [6807981] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 01:20:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807981/ [6807982] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807982/ [6807983] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:21:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807983/ [6807984] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:17 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807984/ [6807985] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:11:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807985/ [6807986] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:19:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807986/ [6807987] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} time: 00:14:57 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807987/ [6807988] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:17:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807988/ [6807989] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:22:23 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807989/ [6807990] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807990/ [6807991] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:27:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807991/ [6807992] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:12:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807992/ [6807993] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:10:17 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807993/ [6807994] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 01:27:08 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807994/ [6807995] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:07:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807995/ [6807996] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807996/ [6807997] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:18:36 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807997/ [6807998] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:33:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807998/ [6807999] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6807999/ [6808000] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:37:01 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808000/ [6808001] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808001/ [6808002] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:27:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808002/ [6808003] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808003/ [6808004] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808004/ [6808005] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:42 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808005/ [6808007] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808007/ [6808008] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:34 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808008/ [6808009] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:24:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808009/ [6808010] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:07:04 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808010/ [6808011] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:09:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808011/ [6808012] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808012/ [6808013] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808013/ [6808014] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:39:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808014/ [6808016] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:14:58 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808016/ [6808017] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:21:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808017/ [6808018] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808018/ [6808019] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:30:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808019/ [6808020] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:10:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808020/ [6808021] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808021/ [6808022] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:16:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808022/ [6808023] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:18:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808023/ [6808024] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808024/ [6808025] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808025/ [6808026] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808026/ [6808027] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:25:14 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808027/ [6808028] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 01:00:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808028/ [6808029] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808029/ [6808030] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:30:07 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808030/ [6808031] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:27:45 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808031/ [6808032] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:51:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808032/ [6808033] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:20:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808033/ [6808034] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/workunits} time: 00:19:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808034/ [6808035] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808035/ [6808036] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} time: 00:31:10 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808036/ [6808037] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808037/ [6808038] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:33:30 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808038/ [6808039] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:17:22 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808039/ [6808040] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:21:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808040/ [6808041] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:33:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808041/ [6808042] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808042/ [6808043] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808043/ [6808045] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:25 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808045/ [6808046] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:06 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808046/ [6808047] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:11:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808047/ [6808048] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:19:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808048/ [6808049] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects} time: 00:24:29 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808049/ [6808050] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808050/ [6808051] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:38:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808051/ [6808052] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:13:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808052/ [6808053] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:19:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808053/ [6808054] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:28:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808054/ [6808055] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:35:19 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808055/ [6808056] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:15:28 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808056/ [6808057] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:17:31 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808057/ [6808058] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808058/ [6808059] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:38:33 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808059/ [6808060] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808060/ [6808061] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:27 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808061/ [6808062] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:14:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808062/ [6808063] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:30:09 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808063/ [6808064] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808064/ [6808065] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:20:47 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808065/ [6808066] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:48:08 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808066/ [6808067] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:19:21 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808067/ [6808068] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:25:50 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808068/ [6808069] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808069/ [6808070] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:24:16 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808070/ [6808071] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808071/ [6808072] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:35:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808072/ [6808073] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:37:37 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808073/ [6808074] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808074/ [6808075] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:10:32 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808075/ [6808076] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/crash} time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808076/ [6808077] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:39:40 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808077/ [6808079] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:42:38 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808079/ [6808080] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:16:49 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808080/ [6808081] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:52:11 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808081/ [6808082] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:24:29 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808082/ [6808083] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:26:20 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808083/ [6808084] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:00 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808084/ [6808085] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808085/ [6808086] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:19:29 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808086/ [6808087] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:20:02 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808087/ [6808088] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:40:18 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808088/ [6808090] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:52 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808090/ [6808091] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:26:35 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808091/ [6808092] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:29:13 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808092/ [6808093] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:28:15 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808093/ [6808094] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:48 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808094/ [6808095] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:41:45 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808095/ [6808097] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:19:03 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808097/ [6808098] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:32 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808098/ [6808099] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:15:41 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808099/ [6808100] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:24 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808100/ [6808101] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:30:51 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808101/ [6808102] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} time: 00:31:12 info: http://pulpito.ceph.com/yuriw-2022-04-27_02:52:22-rados-pacific-distro-default-smithi/6808102/