2023-07-25T16:06:59.700 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-25T16:06:59.701 INFO:teuthology.results:subset: '111/120000' 2023-07-25T16:06:59.701 INFO:teuthology.results:seed: '4711' 2023-07-26T01:10:30.503 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-26T01:10:30.541 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-07-26T07:59:23.143 INFO:teuthology.results:Tests finished! gathering results... 2023-07-26T07:59:47.684 INFO:scrape:Found 305 jobs 2023-07-26T07:59:47.686 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351339/teuthology.log 2023-07-26T07:59:48.942 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:48.943 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:24:24.090 85994== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.080 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.083 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.944 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:26:04.429 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.399 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.432 85992== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.945 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:27:49.714 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.754 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.726 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.950 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:48.951 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:24:24.090 85994== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.080 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.083 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.952 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:26:04.429 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.399 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.432 85992== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.953 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:27:49.714 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.754 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.726 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.960 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:48.961 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:24:24.090 85994== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.080 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.083 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.961 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:26:04.429 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.399 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.432 85992== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.962 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:27:49.714 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.754 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.726 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.967 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:48.968 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:24:24.090 85994== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.080 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:24:24.083 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.968 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:26:04.429 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.399 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:26:04.432 85992== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:48.969 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:27:49.714 85009== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.754 85992== Warning: unimplemented fcntl command: 1036 , ==00:00:27:49.726 85009== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:49.655 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:49.658 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x562336] , 2: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, std::chrono::time_point > >)+0x25a) [0xb9961a] 2023-07-26T07:59:49.659 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:19:28.082 84926== Warning: unimplemented fcntl command: 1036 , ==00:00:19:28.084 84926== Warning: unimplemented fcntl command: 1036 , ==00:00:19:28.109 84926== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:49.659 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:20:53.478 85070== Warning: unimplemented fcntl command: 1036 , ==00:00:20:53.498 84927== Warning: unimplemented fcntl command: 1036 , ==00:00:20:53.988 84926== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:49.660 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:21:53.675 84927== Warning: unimplemented fcntl command: 1036 , 2023-07-26T02:14:38.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~224s , 2023-07-26T02:14:45.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~230s 2023-07-26T07:59:49.688 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0] , 2: /lib64/libc.so.6(+0xa154c) [0x54e954c] 2023-07-26T07:59:49.688 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.1.2-175-gc1444501 (c1444501ab7918ce42bdc26b9d860ad26e34dd69) reef (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x562336] , 2: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, std::chrono::time_point > >)+0x25a) [0xb9961a] 2023-07-26T07:59:49.689 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:19:28.082 84926== Warning: unimplemented fcntl command: 1036 , ==00:00:19:28.084 84926== Warning: unimplemented fcntl command: 1036 , ==00:00:19:28.109 84926== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:49.689 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:20:53.478 85070== Warning: unimplemented fcntl command: 1036 , ==00:00:20:53.498 84927== Warning: unimplemented fcntl command: 1036 , ==00:00:20:53.988 84926== Warning: unimplemented fcntl command: 1036 2023-07-26T07:59:49.690 WARNING:scrape:Ignoring malparsed backtrace: ==00:00:21:53.675 84927== Warning: unimplemented fcntl command: 1036 , 2023-07-26T02:14:38.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~224s , 2023-07-26T02:14:45.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~230s 2023-07-26T07:59:49.866 INFO:scrape:Found 8 distinct failure reasons 2023-07-26T07:59:49.867 INFO:scrape:Dead: None 2023-07-26T07:59:49.867 INFO:scrape:26 jobs: ['7351339', '7351232', '7351263', '7351319', '7351230', '7351351', '7351309', '7351236', '7351286', '7351237', '7351352', '7351287', '7351308', '7351306', '7351358', '7351288', '7351285', '7351289', '7351278', '7351372', '7351432', '7351354', '7351219', '7351216', '7351378', '7351353'] 2023-07-26T07:59:49.868 INFO:scrape: 2023-07-26T07:59:49.868 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-07-26T07:59:49.868 INFO:scrape:2 jobs: ['7351248', '7351415'] 2023-07-26T07:59:49.869 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-07-26T07:59:49.869 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2023-07-26T07:59:49.869 INFO:scrape: 2023-07-26T07:59:49.870 INFO:scrape:Failure: "2023-07-26T01:30:28.895896+0000 mon.a (mon.0) 254 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2023-07-26T07:59:49.870 INFO:scrape:1 jobs: ['7351357'] 2023-07-26T07:59:49.870 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2023-07-26T07:59:49.871 INFO:scrape: 2023-07-26T07:59:49.871 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/rados/test.sh 2023-07-26T07:59:49.871 INFO:scrape:2 jobs: ['7351171', '7351426'] 2023-07-26T07:59:49.872 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2023-07-26T07:59:49.872 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2023-07-26T07:59:49.872 INFO:scrape: 2023-07-26T07:59:49.873 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2023-07-26T07:59:49.873 INFO:scrape:1 jobs: ['7350979'] 2023-07-26T07:59:49.873 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2023-07-26T07:59:49.874 INFO:scrape: 2023-07-26T07:59:49.874 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2023-07-26T07:59:49.874 INFO:scrape:1 jobs: ['7351265'] 2023-07-26T07:59:49.875 INFO:scrape:suites: ['mon_election/classic', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{centos_latest}}'] 2023-07-26T07:59:49.875 INFO:scrape: 2023-07-26T07:59:49.875 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi171.front.sepia.ceph.com: ['type=AVC msg=audit(1690334134.003:20780): avc: denied { ioctl } for pid=113002 comm="iptables" path="/var/lib/containers/storage/overlay/36ed0f752980272c3cad1ec2435f0b8c805a1a70e2475fa813d91bb3c63a12e1/merged" dev="overlay" ino=3408738 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1690334133.934:20779): avc: denied { ioctl } for pid=112998 comm="iptables" path="/var/lib/containers/storage/overlay/36ed0f752980272c3cad1ec2435f0b8c805a1a70e2475fa813d91bb3c63a12e1/merged" dev="overlay" ino=3408738 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2023-07-26T07:59:49.876 INFO:scrape:1 jobs: ['7351393'] 2023-07-26T07:59:49.876 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_cephadm}'] 2023-07-26T07:59:49.876 INFO:scrape: 2023-07-26T07:59:49.876 INFO:scrape:Failure: Command failed on smithi043 with status 1: 'sudo yum install -y kernel' 2023-07-26T07:59:49.877 INFO:scrape:1 jobs: ['7351212'] 2023-07-26T07:59:49.877 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/fastclose', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{more-async-partial-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}'] 2023-07-26T07:59:49.877 INFO:scrape: 2023-07-26T07:59:49.888 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/ failed: 9 dead: 26 running: 0 waiting: 0 queued: 0 passed: 270 Fail ================================================================= [7350979] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 04:08:52 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350979/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350979/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fa5a8dc5bc904d48a6af2072128e70aa Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi175 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh' [7351171] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:55:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351171/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351171/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5daf8da8f64b4a4187020bf699bf39f7 Command failed (workunit test rados/test.sh) on smithi153 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7351212] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351212/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351212/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1840fffb5e94358ae05fa8eebe1b25d Command failed on smithi043 with status 1: 'sudo yum install -y kernel' [7351248] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:40:39 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351248/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351248/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aadbf327200f4148ac13a425961a39c6 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7351265] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351265/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351265/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a6d87f75a7da4636b5e6b0566a776ffa Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7351357] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 02:45:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351357/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351357/ "2023-07-26T01:30:28.895896+0000 mon.a (mon.0) 254 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [7351393] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351393/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351393/ SELinux denials found on ubuntu@smithi171.front.sepia.ceph.com: ['type=AVC msg=audit(1690334134.003:20780): avc: denied { ioctl } for pid=113002 comm="iptables" path="/var/lib/containers/storage/overlay/36ed0f752980272c3 cad1ec2435f0b8c805a1a70e2475fa813d91bb3c63a12e1/merged" dev="overlay" ino=3408738 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1690334133.934:20779): avc: denied { ioctl } for pid=112998 comm="iptables" path="/var/lib/containers/storage/o verlay/36ed0f752980272c3cad1ec2435f0b8c805a1a70e2475fa813d91bb3c63a12e1/mer ged" dev="overlay" ino=3408738 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7351415] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:37:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351415/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351415/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad1873ffb3db48e68f62b8541aad5620 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7351426] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:45:39 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351426/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351426/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b47ed75d4fef4e9a854a394525b2f081 Command failed (workunit test rados/test.sh) on smithi017 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1444501ab7918ce42bdc26b9d860ad26e34dd69 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' Dead ================================================================= [7351216] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351216/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351216/ Error reimaging machines: Failed to power on smithi204 [7351219] rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351219/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351219/ Error reimaging machines: Expected smithi188's OS to be centos 9 but found ubuntu 22.04 [7351230] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351230/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351230/ Error reimaging machines: Expected smithi090's OS to be rhel 8.6 but found centos 9 [7351232] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351232/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351232/ Error reimaging machines: Expected smithi082's OS to be rhel 8.6 but found centos 8 [7351236] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351236/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351236/ Error reimaging machines: Expected smithi194's OS to be centos 8 but found centos 9 [7351237] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351237/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351237/ Error reimaging machines: Expected smithi146's OS to be rhel 8.6 but found centos 8 [7351263] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351263/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351263/ Error reimaging machines: Failed to power on smithi136 [7351278] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351278/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351278/ Error reimaging machines: Failed to power on smithi144 [7351285] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351285/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351285/ Error reimaging machines: Failed to power on smithi157 [7351286] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351286/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351286/ Error reimaging machines: Cannot connect to remote host smithi049 [7351287] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351287/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351287/ Error reimaging machines: Failed to power on smithi144 [7351288] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351288/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351288/ Error reimaging machines: Failed to power on smithi005 [7351289] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351289/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351289/ Error reimaging machines: Expected smithi155's OS to be centos 9 but found centos 8 [7351306] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351306/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351306/ Error reimaging machines: Failed to power on smithi086 [7351308] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351308/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351308/ Error reimaging machines: Expected smithi099's OS to be ubuntu 22.04 but found ubuntu 20.04 [7351309] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351309/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351309/ Error reimaging machines: Cannot connect to remote host smithi090 [7351319] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351319/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351319/ Error reimaging machines: Failed to power on smithi031 [7351339] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351339/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351339/ Error reimaging machines: Failed to power on smithi087 [7351351] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351351/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351351/ Error reimaging machines: Failed to power on smithi204 [7351352] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351352/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351352/ Error reimaging machines: Cannot connect to remote host smithi079 [7351353] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351353/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351353/ Error reimaging machines: Cannot connect to remote host smithi083 [7351354] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351354/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351354/ Error reimaging machines: Expected smithi135's OS to be centos 8 but found ubuntu 20.04 [7351358] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_latest} tasks/prometheus} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351358/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351358/ Error reimaging machines: Failed to power on smithi040 [7351372] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351372/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351372/ Error reimaging machines: Failed to power on smithi145 [7351378] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351378/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351378/ Error reimaging machines: Failed to power on smithi097 [7351432] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351432/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351432/ Error reimaging machines: Failed to power on smithi107 Pass ================================================================= [7350937] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:25:27 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350937/ [7350938] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:43 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350938/ [7350942] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350942/ [7350945] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:30:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350945/ [7350950] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:19:16 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350950/ [7350953] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_8}} time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350953/ [7350955] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:34:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350955/ [7350956] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350956/ [7350959] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350959/ [7350960] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350960/ [7350963] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache} time: 00:19:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350963/ [7350965] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350965/ [7350967] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350967/ [7350971] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:26:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350971/ [7350975] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:18:53 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350975/ [7350976] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:31:18 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350976/ [7350982] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:16:17 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350982/ [7350985] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350985/ [7350987] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 03:16:29 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350987/ [7350992] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:22:47 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350992/ [7350993] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:39:29 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350993/ [7350996] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350996/ [7350999] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:17:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7350999/ [7351002] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:45:20 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351002/ [7351003] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:21:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351003/ [7351006] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_workunits} time: 00:41:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351006/ [7351009] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:36:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351009/ [7351012] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351012/ [7351017] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:23:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351017/ [7351020] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351020/ [7351022] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:31:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351022/ [7351025] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351025/ [7351029] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:34:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351029/ [7351032] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:19:03 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351032/ [7351035] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:33:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351035/ [7351038] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:31:31 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351038/ [7351041] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:21:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351041/ [7351043] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:20:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351043/ [7351045] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:55 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351045/ [7351048] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:25:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351048/ [7351051] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:36:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351051/ [7351054] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351054/ [7351057] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:43:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351057/ [7351060] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351060/ [7351064] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:41 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351064/ [7351068] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:43:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351068/ [7351071] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:17:51 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351071/ [7351073] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351073/ [7351076] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:44:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351076/ [7351079] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:27:05 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351079/ [7351083] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:23:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351083/ [7351086] rados/standalone/{supported-random-distro$/{centos_latest} workloads/c2c} time: 00:19:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351086/ [7351089] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:39:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351089/ [7351092] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:34:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351092/ [7351095] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351095/ [7351097] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:28:19 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351097/ [7351102] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351102/ [7351106] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:26:14 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351106/ [7351111] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:33:34 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351111/ [7351117] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:15:51 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351117/ [7351120] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351120/ [7351122] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:15:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351122/ [7351123] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:37:33 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351123/ [7351125] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:57 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351125/ [7351127] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:23:12 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351127/ [7351129] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351129/ [7351131] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:11:15 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351131/ [7351133] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench} time: 00:28:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351133/ [7351135] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:25:23 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351135/ [7351137] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351137/ [7351139] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:04 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351139/ [7351141] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:52 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351141/ [7351143] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:17:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351143/ [7351145] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351145/ [7351147] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:34:57 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351147/ [7351150] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351150/ [7351152] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351152/ [7351154] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:59 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351154/ [7351156] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351156/ [7351157] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:34:03 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351157/ [7351159] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351159/ [7351161] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351161/ [7351163] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351163/ [7351165] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:38 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351165/ [7351167] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:23:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351167/ [7351169] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:37:59 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351169/ [7351173] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:23:32 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351173/ [7351175] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351175/ [7351177] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:31:45 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351177/ [7351178] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:28:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351178/ [7351180] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351180/ [7351183] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:21:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351183/ [7351184] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:28:58 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351184/ [7351186] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_latest}} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351186/ [7351189] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/sync-many workloads/pool-create-delete} time: 00:21:57 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351189/ [7351191] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:20:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351191/ [7351192] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:27:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351192/ [7351194] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:13:23 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351194/ [7351196] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:21:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351196/ [7351198] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351198/ [7351200] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351200/ [7351202] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:19:52 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351202/ [7351203] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/insights} time: 00:16:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351203/ [7351206] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351206/ [7351208] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:28:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351208/ [7351210] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351210/ [7351214] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:26:19 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351214/ [7351218] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:29:43 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351218/ [7351222] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:35 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351222/ [7351224] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351224/ [7351226] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351226/ [7351228] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:39:27 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351228/ [7351234] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:28:40 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351234/ [7351240] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:22:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351240/ [7351242] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:16:43 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351242/ [7351244] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:19:20 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351244/ [7351246] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:48 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351246/ [7351251] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:16 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351251/ [7351252] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:33:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351252/ [7351253] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:33:11 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351253/ [7351256] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/rados_5925} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351256/ [7351258] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:15:17 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351258/ [7351260] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:13:33 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351260/ [7351261] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:13:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351261/ [7351268] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_20.04} tasks/module_selftest} time: 00:29:10 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351268/ [7351270] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:34:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351270/ [7351271] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:20:24 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351271/ [7351274] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351274/ [7351276] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:20:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351276/ [7351277] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:18:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351277/ [7351279] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:09:34 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351279/ [7351280] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:21:55 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351280/ [7351281] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:13:40 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351281/ [7351282] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:48 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351282/ [7351283] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:32:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351283/ [7351284] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:18:33 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351284/ [7351290] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:34:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351290/ [7351291] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351291/ [7351292] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:31:45 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351292/ [7351293] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_big} time: 00:38:10 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351293/ [7351294] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:53 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351294/ [7351295] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351295/ [7351296] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:27:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351296/ [7351297] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} time: 00:40:43 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351297/ [7351298] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:32:46 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351298/ [7351299] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:24:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351299/ [7351300] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/per_module_finisher_stats} time: 00:15:48 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351300/ [7351301] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:27:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351301/ [7351302] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:24:24 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351302/ [7351303] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351303/ [7351304] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351304/ [7351305] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351305/ [7351307] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:19:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351307/ [7351310] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351310/ [7351311] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:19:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351311/ [7351312] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:24:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351312/ [7351313] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351313/ [7351314] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mix} time: 00:31:18 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351314/ [7351315] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_latest}} time: 00:36:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351315/ [7351316] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:23:02 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351316/ [7351317] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:26:20 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351317/ [7351318] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351318/ [7351320] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:38:09 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351320/ [7351321] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:39:40 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351321/ [7351322] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:18:12 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351322/ [7351323] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:18:45 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351323/ [7351324] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351324/ [7351325] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:26:02 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351325/ [7351326] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:23:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351326/ [7351327] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:41:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351327/ [7351328] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/progress} time: 00:38:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351328/ [7351329] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:41:37 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351329/ [7351330] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351330/ [7351331] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351331/ [7351332] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:04:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351332/ [7351333] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:13:58 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351333/ [7351334] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:30:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351334/ [7351335] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351335/ [7351336] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:19:32 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351336/ [7351337] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:32:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351337/ [7351338] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:18:04 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351338/ [7351340] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon-stretch} time: 00:26:09 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351340/ [7351341] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:41:08 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351341/ [7351342] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:19:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351342/ [7351343] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:17:36 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351343/ [7351344] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:34:02 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351344/ [7351345] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351345/ [7351346] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351346/ [7351347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:25:06 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351347/ [7351348] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:58 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351348/ [7351349] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:15:05 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351349/ [7351350] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351350/ [7351355] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:07 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351355/ [7351356] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:24:24 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351356/ [7351359] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 01:24:27 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351359/ [7351360] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351360/ [7351361] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{rhel_8}} time: 02:22:09 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351361/ [7351362] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:23:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351362/ [7351363] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/readwrite} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351363/ [7351364] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351364/ [7351365] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:16:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351365/ [7351366] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:35:22 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351366/ [7351367] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:23:35 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351367/ [7351368] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:33:27 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351368/ [7351369] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon} time: 01:11:57 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351369/ [7351370] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351370/ [7351371] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:22:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351371/ [7351373] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351373/ [7351374] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351374/ [7351375] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:41 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351375/ [7351376] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351376/ [7351377] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:21:41 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351377/ [7351379] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:26:08 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351379/ [7351380] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351380/ [7351381] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:36:47 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351381/ [7351382] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:23:51 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351382/ [7351383] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:21:47 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351383/ [7351384] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:37:32 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351384/ [7351385] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351385/ [7351386] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:21:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351386/ [7351387] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:20:30 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351387/ [7351388] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:27:32 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351388/ [7351389] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351389/ [7351390] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 01:01:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351390/ [7351391] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:31:23 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351391/ [7351392] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:12:15 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351392/ [7351394] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351394/ [7351395] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:34:39 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351395/ [7351396] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351396/ [7351397] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:15:27 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351397/ [7351398] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351398/ [7351399] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd-backfill} time: 02:46:50 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351399/ [7351400] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:53:17 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351400/ [7351401] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:28:18 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351401/ [7351402] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351402/ [7351403] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351403/ [7351404] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351404/ [7351405] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:43:03 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351405/ [7351406] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:25:36 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351406/ [7351407] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:25:55 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351407/ [7351408] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351408/ [7351409] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:20:13 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351409/ [7351410] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351410/ [7351411] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:23:34 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351411/ [7351412] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:41:39 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351412/ [7351413] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:25 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351413/ [7351414] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:32:58 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351414/ [7351416] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/crash} time: 00:17:56 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351416/ [7351417] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:35 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351417/ [7351418] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:24:48 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351418/ [7351419] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:31:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351419/ [7351420] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:29:16 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351420/ [7351421] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:23 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351421/ [7351422] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:17:48 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351422/ [7351423] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:29:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351423/ [7351424] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:57:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351424/ [7351425] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:15 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351425/ [7351427] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:18:12 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351427/ [7351428] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:19:28 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351428/ [7351429] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:16:55 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351429/ [7351430] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd} time: 02:44:54 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351430/ [7351431] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/libcephsqlite} time: 00:24:15 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351431/ [7351433] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:24:52 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351433/ [7351434] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351434/ [7351435] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/rados_5925} time: 00:14:42 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351435/ [7351436] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:37:17 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351436/ [7351437] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:16:32 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351437/ [7351438] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351438/ [7351439] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_nfs} time: 00:38:01 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351439/ [7351440] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-07-25_14:29:32-rados-reef-release-distro-default-smithi/7351440/