2023-06-14T09:36:48.323 INFO:root:teuthology version: 0.0.1.dev122+gb580191 2023-06-14T09:36:48.324 INFO:teuthology.results:subset: '1/300000' 2023-06-14T09:36:48.324 INFO:teuthology.results:seed: '2550' 2023-06-15T04:44:50.826 INFO:root:teuthology version: 0.0.1.dev124+g961f4fb 2023-06-15T04:44:50.836 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-15T08:26:12.263 INFO:teuthology.results:Tests finished! gathering results... 2023-06-15T08:26:34.510 INFO:scrape:Found 304 jobs 2023-06-15T08:26:34.913 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.0.0-4128-g6fc43019 (6fc43019e0026c4a1ddd83a3fab6fb8c23187b49) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fec1a6826e1] , 2: /usr/lib/ceph/libceph-common.so.2(+0x161895) [0x7fec1a682895] 2023-06-15T08:26:34.915 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:47:25.278 INFO:tasks.ceph:Scrubbing osd.5 , 2023-06-15T04:47:25.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 , 2023-06-15T04:47:25.423 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-06-15T08:26:34.916 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:48:54.029 INFO:tasks.ceph.osd.3:Stopped , 2023-06-15T04:48:54.029 DEBUG:tasks.ceph.osd.4:waiting for process to exit , 2023-06-15T04:48:54.030 INFO:teuthology.orchestra.run:waiting for 300 2023-06-15T08:26:34.916 WARNING:scrape:Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.33864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33864.log.gz , /var/log/ceph/ceph-client.admin.36342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2023-06-15T08:26:34.917 WARNING:scrape:Ignoring malparsed backtrace: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz , /var/log/ceph/ceph-client.admin.37924.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log , -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2023-06-15T08:26:34.917 WARNING:scrape:Ignoring malparsed backtrace: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise , raise value.with_traceback(tb) 2023-06-15T08:26:34.918 WARNING:scrape:Ignoring malparsed backtrace: self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/teuthology/orchestra/run.py", line 179, in _raise_for_status , raise CommandCrashedError(command=self.command) 2023-06-15T08:26:35.082 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.0.0-4128-g6fc43019 (6fc43019e0026c4a1ddd83a3fab6fb8c23187b49) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fec1a6826e1] , 2: /usr/lib/ceph/libceph-common.so.2(+0x161895) [0x7fec1a682895] 2023-06-15T08:26:35.084 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:47:25.278 INFO:tasks.ceph:Scrubbing osd.5 , 2023-06-15T04:47:25.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 , 2023-06-15T04:47:25.423 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-06-15T08:26:35.084 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:48:54.029 INFO:tasks.ceph.osd.3:Stopped , 2023-06-15T04:48:54.029 DEBUG:tasks.ceph.osd.4:waiting for process to exit , 2023-06-15T04:48:54.030 INFO:teuthology.orchestra.run:waiting for 300 2023-06-15T08:26:35.085 WARNING:scrape:Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.33864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33864.log.gz , /var/log/ceph/ceph-client.admin.36342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2023-06-15T08:26:35.085 WARNING:scrape:Ignoring malparsed backtrace: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz , /var/log/ceph/ceph-client.admin.37924.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log , -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2023-06-15T08:26:35.086 WARNING:scrape:Ignoring malparsed backtrace: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise , raise value.with_traceback(tb) 2023-06-15T08:26:35.086 WARNING:scrape:Ignoring malparsed backtrace: self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/teuthology/orchestra/run.py", line 179, in _raise_for_status , raise CommandCrashedError(command=self.command) 2023-06-15T08:26:35.274 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.0.0-4128-g6fc43019 (6fc43019e0026c4a1ddd83a3fab6fb8c23187b49) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fec1a6826e1] , 2: /usr/lib/ceph/libceph-common.so.2(+0x161895) [0x7fec1a682895] 2023-06-15T08:26:35.276 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:47:25.278 INFO:tasks.ceph:Scrubbing osd.5 , 2023-06-15T04:47:25.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 , 2023-06-15T04:47:25.423 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-06-15T08:26:35.277 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:48:54.029 INFO:tasks.ceph.osd.3:Stopped , 2023-06-15T04:48:54.029 DEBUG:tasks.ceph.osd.4:waiting for process to exit , 2023-06-15T04:48:54.030 INFO:teuthology.orchestra.run:waiting for 300 2023-06-15T08:26:35.277 WARNING:scrape:Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.33864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33864.log.gz , /var/log/ceph/ceph-client.admin.36342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2023-06-15T08:26:35.278 WARNING:scrape:Ignoring malparsed backtrace: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz , /var/log/ceph/ceph-client.admin.37924.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log , -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2023-06-15T08:26:35.278 WARNING:scrape:Ignoring malparsed backtrace: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise , raise value.with_traceback(tb) 2023-06-15T08:26:35.279 WARNING:scrape:Ignoring malparsed backtrace: self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/teuthology/orchestra/run.py", line 179, in _raise_for_status , raise CommandCrashedError(command=self.command) 2023-06-15T08:26:35.453 WARNING:scrape:Ignoring malparsed backtrace: ceph version 18.0.0-4128-g6fc43019 (6fc43019e0026c4a1ddd83a3fab6fb8c23187b49) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fec1a6826e1] , 2: /usr/lib/ceph/libceph-common.so.2(+0x161895) [0x7fec1a682895] 2023-06-15T08:26:35.455 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:47:25.278 INFO:tasks.ceph:Scrubbing osd.5 , 2023-06-15T04:47:25.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 , 2023-06-15T04:47:25.423 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-06-15T08:26:35.456 WARNING:scrape:Ignoring malparsed backtrace: 2023-06-15T04:48:54.029 INFO:tasks.ceph.osd.3:Stopped , 2023-06-15T04:48:54.029 DEBUG:tasks.ceph.osd.4:waiting for process to exit , 2023-06-15T04:48:54.030 INFO:teuthology.orchestra.run:waiting for 300 2023-06-15T08:26:35.456 WARNING:scrape:Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.33864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33864.log.gz , /var/log/ceph/ceph-client.admin.36342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2023-06-15T08:26:35.457 WARNING:scrape:Ignoring malparsed backtrace: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz , /var/log/ceph/ceph-client.admin.37924.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log , -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2023-06-15T08:26:35.457 WARNING:scrape:Ignoring malparsed backtrace: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 65, in reraise , raise value.with_traceback(tb) 2023-06-15T08:26:35.458 WARNING:scrape:Ignoring malparsed backtrace: self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_teuthology_b580191f840dede6fc19423f098cbe32f5ed42bf/teuthology/orchestra/run.py", line 179, in _raise_for_status , raise CommandCrashedError(command=self.command) 2023-06-15T08:26:35.478 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303746/teuthology.log 2023-06-15T08:26:38.308 INFO:scrape:Found 11 distinct failure reasons 2023-06-15T08:26:38.309 INFO:scrape:Failure: {'smithi178.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.353401', 'end': '2023-06-14 20:36:49.233877', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:36:35.880476', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi006.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:11.576510', 'end': '2023-06-14 20:36:49.248662', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:36:37.672152', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} 2023-06-15T08:26:38.310 INFO:scrape:8 jobs: ['7303762', '7303795', '7303810', '7303745', '7303763', '7303821', '7303803', '7303807'] 2023-06-15T08:26:38.310 INFO:scrape:suites intersection: [] 2023-06-15T08:26:38.310 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-inject-leak/osd', '2-ops/rm-zap-add}', 'agent/off', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'rados/singleton-nomsgr/{all/cache-fs-trunc', 'rados/singleton-nomsgr/{all/health-warnings', 'rados/singleton/{all/resolve_stuck_peering', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}}', 'task/test_iscsi_container/{centos_8.stream_container_tools', 'tasks/rados_cls_all', 'tasks/scrub_test}', 'test_iscsi_container}}', 'thrashosds-health}', 'validater/lockdep}'] 2023-06-15T08:26:38.311 INFO:scrape: 2023-06-15T08:26:38.311 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 2023-06-15T08:26:38.311 INFO:scrape:1 jobs: ['7303854'] 2023-06-15T08:26:38.312 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{ubuntu_latest}}'] 2023-06-15T08:26:38.312 INFO:scrape: 2023-06-15T08:26:38.312 INFO:scrape:Dead: None 2023-06-15T08:26:38.312 INFO:scrape:13 jobs: ['7303746', '7303740', '7303752', '7303750', '7303767', '7303760', '7303751', '7303804', '7303793', '7303761', '7303747', '7303741', '7303826'] 2023-06-15T08:26:38.313 INFO:scrape: 2023-06-15T08:26:38.313 INFO:scrape:Failure: Command failed on smithi115 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" 2023-06-15T08:26:38.313 INFO:scrape:2 jobs: ['7303806', '7303830'] 2023-06-15T08:26:38.314 INFO:scrape:suites intersection: ['rados'] 2023-06-15T08:26:38.314 INFO:scrape:suites union: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/normal', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/careful', 'thrashosds-health', 'three-plus-one}', 'workloads/rbd_cls}'] 2023-06-15T08:26:38.314 INFO:scrape: 2023-06-15T08:26:38.314 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi062 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-06-15T08:26:38.315 INFO:scrape:2 jobs: ['7303870', '7303823'] 2023-06-15T08:26:38.315 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/none', 'msgr/async', 'rados', 'test_envlibrados_for_rocksdb}'] 2023-06-15T08:26:38.315 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/none', 'msgr/async', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{ubuntu_20.04}}', 'supported-random-distro$/{ubuntu_latest}}', 'test_envlibrados_for_rocksdb}'] 2023-06-15T08:26:38.316 INFO:scrape: 2023-06-15T08:26:38.316 INFO:scrape:Dead: b'2023-06-15T08:22:31.165 DEBUG:teuthology.exit:Finished running handlers' 2023-06-15T08:26:38.316 INFO:scrape:3 jobs: ['7303742', '7303654', '7303595'] 2023-06-15T08:26:38.317 INFO:scrape:suites intersection: [] 2023-06-15T08:26:38.317 INFO:scrape:suites union: ['debug/mgr', 'mgr_ttl_cache/enable', 'mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/singleton-nomsgr/{all/admin_socket_output', 'random-objectstore$/{bluestore-stupid}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_20.04}}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/progress}'] 2023-06-15T08:26:38.317 INFO:scrape: 2023-06-15T08:26:38.317 INFO:scrape:Failure: Stale jobs detected, aborting. 2023-06-15T08:26:38.318 INFO:scrape:1 jobs: ['7303749'] 2023-06-15T08:26:38.318 INFO:scrape:suites: ['0-nvme-loop', 'agent/off', 'fixed-2', 'mon_election/classic', 'rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'start}'] 2023-06-15T08:26:38.318 INFO:scrape: 2023-06-15T08:26:38.319 INFO:scrape:Failure: Command failed on smithi105 with status 1: 'sudo yum -y install ceph-mgr-dashboard' 2023-06-15T08:26:38.319 INFO:scrape:1 jobs: ['7303824'] 2023-06-15T08:26:38.319 INFO:scrape:suites: ['cluster/{12-osds', 'mon_election/classic', 'msgr-failures/fastclose', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2023-06-15T08:26:38.320 INFO:scrape: 2023-06-15T08:26:38.320 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi125 with status 4: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-06-15T08:26:38.320 INFO:scrape:2 jobs: ['7303818', '7303667'] 2023-06-15T08:26:38.320 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-06-15T08:26:38.321 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-lz4}', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'tasks/e2e}'] 2023-06-15T08:26:38.321 INFO:scrape: 2023-06-15T08:26:38.321 INFO:scrape:Failure: Command failed on smithi012 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 2023-06-15T08:26:38.322 INFO:scrape:3 jobs: ['7303668', '7303594', '7303819'] 2023-06-15T08:26:38.322 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04'] 2023-06-15T08:26:38.322 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/flannel', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] 2023-06-15T08:26:38.323 INFO:scrape: 2023-06-15T08:26:38.323 INFO:scrape:Failure: No module named 'tasks' 2023-06-15T08:26:38.323 INFO:scrape:1 jobs: ['7303636'] 2023-06-15T08:26:38.324 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rmdir-reactivate}', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0'] 2023-06-15T08:26:38.324 INFO:scrape: 2023-06-15T08:26:38.335 INFO:teuthology.results:Sending results to chunmei.liu@intel.com: Test Run: yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/ failed: 13 dead: 24 running: 0 waiting: 0 queued: 0 passed: 267 Fail ================================================================= [7303594] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303594/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303594/ sentry: https://sentry.ceph.com/organizations/ceph/?query=663e1347bee34b70aafdc35bde38895f Command failed on smithi182 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7303636] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:09:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303636/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303636/ sentry: https://sentry.ceph.com/organizations/ceph/?query=373adfde477647a99af260e0ecc929f3 No module named 'tasks' [7303667] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e} ----------------------------------------------------------------- time: 00:29:16 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303667/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303667/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5567bbf80b444d1cb17ddcc39fc9dac0 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi072 with status 4: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7303668] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:06:15 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303668/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303668/ sentry: https://sentry.ceph.com/organizations/ceph/?query=790b88623aae481bbcf5eb0f2cdc32c3 Command failed on smithi012 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7303749] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303749/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303749/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e316a40c1ca4afea7eb85252bb9422f Stale jobs detected, aborting. [7303806] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:24:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303806/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303806/ sentry: https://sentry.ceph.com/organizations/ceph/?query=de32b24365004ead8a7ef5dd63752b98 Command failed on smithi115 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-Te stClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" [7303818] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/e2e} ----------------------------------------------------------------- time: 00:35:28 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303818/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303818/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3780d18506e4d8abf800f5ab954056b Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi125 with status 4: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7303819] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303819/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303819/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a47d6065a96f4fdb99ce686dd34c351e Command failed on smithi086 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7303823] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:10:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303823/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303823/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a408840d53041a0bf5c45bdcc373e4f Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi174 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7303824] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:12:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303824/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303824/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b955d8eb9fb4b49a2a01956f96f004d Command failed on smithi105 with status 1: 'sudo yum -y install ceph-mgr- dashboard' [7303830] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303830/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303830/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03437323afe54178b131f75f43349f3b Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7303854] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:21:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303854/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303854/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' [7303870] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:12:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303870/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303870/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f905932c33c940c191fe89658f679601 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi062 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fc43019e0026c4a1ddd83a3fab6fb8c23187b49 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' Dead ================================================================= [7303595] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303595/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303595/ hit max job timeout [7303654] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/progress} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303654/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303654/ hit max job timeout [7303740] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/crash} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303740/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303740/ Error reimaging machines: Failed to power on smithi134 [7303741] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303741/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303741/ Error reimaging machines: Failed to power on smithi032 [7303742] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303742/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303742/ hit max job timeout [7303745] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} ----------------------------------------------------------------- time: 00:06:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303745/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303745/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7f3e1b3815a4ebf96a9451467559126 {'smithi071.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.396902', 'end': '2023-06-14 20:35:01.530709', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:34:48.133807', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303746] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/set-chunks-read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303746/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303746/ Error reimaging machines: Failed to power on smithi186 [7303747] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303747/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303747/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds [7303750] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303750/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303750/ Error reimaging machines: Expected smithi079's OS to be ubuntu 22.04 but found centos 8 [7303751] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303751/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303751/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds [7303752] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303752/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303752/ Error reimaging machines: Failed to power on smithi039 [7303760] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303760/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303760/ Error reimaging machines: Failed to power on smithi123 [7303761] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303761/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303761/ Error reimaging machines: Expected smithi047's OS to be ubuntu 22.04 but found centos 8 [7303762] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/lockdep} ----------------------------------------------------------------- time: 00:02:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303762/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303762/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f92aa45753da47958e581af5a45fafff {'smithi178.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.353401', 'end': '2023-06-14 20:36:49.233877', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:36:35.880476', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi006.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:11.576510', 'end': '2023-06-14 20:36:49.248662', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:36:37.672152', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303763] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:06:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303763/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303763/ sentry: https://sentry.ceph.com/organizations/ceph/?query=19d1b7b840d04aa697a2fb6af33c10d0 {'smithi040.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.373570', 'end': '2023-06-14 20:45:19.039498', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:45:05.665928', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303767] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303767/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303767/ Error reimaging machines: Failed to power on smithi115 [7303793] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303793/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303793/ Error reimaging machines: Failed to power on smithi163 [7303795] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} ----------------------------------------------------------------- time: 00:06:44 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303795/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303795/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dac67fc74316451b84e951c4b2b9e981 {'smithi057.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:16.456756', 'end': '2023-06-14 21:09:22.317661', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:09:05.860905', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303803] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:06:15 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303803/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303803/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f96208115d343caa07f2ea6011ae8bc {'smithi136.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:16.430072', 'end': '2023-06-14 21:17:02.848285', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:16:46.418213', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi017.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:21.460322', 'end': '2023-06-14 21:17:17.201029', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:16:55.740707', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303804] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303804/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303804/ Error reimaging machines: Failed to power on smithi182 [7303807] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:13:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303807/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303807/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b6e558a1826245c58a81fcb8c3394251 {'smithi141.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:49.999240', 'end': '2023-06-14 21:19:35.406126', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:18:45.406886', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Network is unreachable]", 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Network is unreachable]"], 'stdout': 'Updating Subscription Management repositories.', 'stdout_lines': ['Updating Subscription Management repositories.']}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303810] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303810/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303810/ sentry: https://sentry.ceph.com/organizations/ceph/?query=08204c2a278a4dd0b2cf6e113dc70780 {'smithi097.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:23.863920', 'end': '2023-06-14 21:18:16.282037', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:17:52.418117', 'stderr': "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nCurl error (7): Couldn't connect to server for https://copr.fe dorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x8 6_64 [Failed to connect to copr.fedorainfracloud.org port 443: Network is unreachable]", 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', "Curl error (7): Couldn't connect to server for https://copr. fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch= x86_64 [Failed to connect to copr.fedorainfracloud.org port 443: Network is unreachable]"], 'stdout': 'Updating Subscription Management repositories.', 'stdout_lines': ['Updating Subscription Management repositories.']}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303821] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/scrub_test} ----------------------------------------------------------------- time: 00:06:19 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303821/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303821/ sentry: https://sentry.ceph.com/organizations/ceph/?query=90d21512c2a24284b7d1d18143008556 {'smithi098.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:21.436299', 'end': '2023-06-14 21:29:07.720788', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:28:46.284489', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi178.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:01.934337', 'end': '2023-06-14 21:29:09.650119', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 21:29:07.715782', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/ coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Network is unreachable', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.or g/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Network is unreachable'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}} [7303826] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303826/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303826/ Error reimaging machines: Failed to power on smithi184 Pass ================================================================= [7303578] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:19:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303578/ [7303579] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303579/ [7303580] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303580/ [7303581] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:39 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303581/ [7303582] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:13 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303582/ [7303583] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:35:25 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303583/ [7303584] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:21:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303584/ [7303585] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:16 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303585/ [7303586] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:26:25 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303586/ [7303587] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:25:50 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303587/ [7303588] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:51:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303588/ [7303589] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:16:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303589/ [7303590] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/valgrind} time: 00:32:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303590/ [7303591] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/dashboard} time: 01:10:19 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303591/ [7303592] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:42:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303592/ [7303593] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:16:59 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303593/ [7303596] rados/standalone/{supported-random-distro$/{rhel_8} workloads/c2c} time: 00:17:14 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303596/ [7303597] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:31:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303597/ [7303598] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:26:49 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303598/ [7303599] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:42 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303599/ [7303600] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:05 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303600/ [7303601] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:16 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303601/ [7303602] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm_repos} time: 00:08:46 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303602/ [7303603] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:11 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303603/ [7303604] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:31:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303604/ [7303605] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303605/ [7303606] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303606/ [7303607] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:15:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303607/ [7303608] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:28:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303608/ [7303609] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:24 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303609/ [7303610] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303610/ [7303611] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:18:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303611/ [7303612] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303612/ [7303613] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303613/ [7303614] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:49 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303614/ [7303615] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:21:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303615/ [7303616] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:52 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303616/ [7303617] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303617/ [7303618] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:15:26 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303618/ [7303619] rados/standalone/{supported-random-distro$/{centos_8} workloads/crush} time: 00:24:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303619/ [7303620] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:25:08 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303620/ [7303621] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303621/ [7303622] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303622/ [7303623] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:41:13 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303623/ [7303624] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:30:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303624/ [7303625] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:15 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303625/ [7303626] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/lockdep} time: 00:32:46 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303626/ [7303627] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:21 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303627/ [7303628] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:25:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303628/ [7303629] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:20:14 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303629/ [7303630] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_nfs} time: 00:31:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303630/ [7303631] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:29:50 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303631/ [7303632] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/per_module_finisher_stats} time: 00:19:39 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303632/ [7303633] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:40:25 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303633/ [7303634] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:19 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303634/ [7303635] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:13:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303635/ [7303637] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:16:58 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303637/ [7303638] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:35:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303638/ [7303639] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:19:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303639/ [7303640] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:14:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303640/ [7303641] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:16 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303641/ [7303642] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:08:34 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303642/ [7303643] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:12:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303643/ [7303644] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli} time: 00:20:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303644/ [7303645] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed} time: 00:14:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303645/ [7303646] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303646/ [7303647] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:32:28 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303647/ [7303648] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 01:28:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303648/ [7303649] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303649/ [7303650] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:52 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303650/ [7303651] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303651/ [7303652] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli_mon} time: 00:36:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303652/ [7303653] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303653/ [7303655] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-radosbench} time: 00:47:25 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303655/ [7303656] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:27:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303656/ [7303657] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/many workloads/snaps-few-objects} time: 00:50:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303657/ [7303658] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:25:47 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303658/ [7303659] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:13:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303659/ [7303660] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/valgrind} time: 01:51:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303660/ [7303661] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303661/ [7303662] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:53 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303662/ [7303663] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps} time: 00:14:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303663/ [7303664] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:15 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303664/ [7303665] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:16 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303665/ [7303666] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:11:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303666/ [7303669] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303669/ [7303670] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:18:35 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303670/ [7303671] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:25:08 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303671/ [7303672] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:31:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303672/ [7303673] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:14:04 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303673/ [7303674] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303674/ [7303675] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:23:21 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303675/ [7303676] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:20:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303676/ [7303677] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_20.04}} time: 00:16:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303677/ [7303678] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:14:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303678/ [7303679] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:17:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303679/ [7303680] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:18:11 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303680/ [7303681] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:33:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303681/ [7303682] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303682/ [7303683] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303683/ [7303684] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:04 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303684/ [7303685] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:22:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303685/ [7303686] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:13:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303686/ [7303687] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:11:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303687/ [7303688] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:15:03 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303688/ [7303689] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303689/ [7303690] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303690/ [7303691] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:07 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303691/ [7303692] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/one workloads/pool-create-delete} time: 00:25:47 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303692/ [7303693] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:09:08 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303693/ [7303694] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/lockdep} time: 00:20:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303694/ [7303695] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:30:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303695/ [7303696] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:31:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303696/ [7303697] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:53:44 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303697/ [7303698] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:20:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303698/ [7303699] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303699/ [7303700] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303700/ [7303701] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption} time: 00:15:44 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303701/ [7303702] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:22:52 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303702/ [7303703] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench} time: 00:59:03 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303703/ [7303704] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_big} time: 00:31:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303704/ [7303705] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:11:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303705/ [7303706] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:26 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303706/ [7303707] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:28:08 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303707/ [7303708] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:15:47 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303708/ [7303709] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 01:10:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303709/ [7303710] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303710/ [7303711] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:20:40 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303711/ [7303712] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_20.04} tasks/workunits} time: 00:10:13 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303712/ [7303713] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:22:24 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303713/ [7303714] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:14:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303714/ [7303715] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:33:25 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303715/ [7303716] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:21:41 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303716/ [7303717] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303717/ [7303718] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303718/ [7303719] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm} time: 00:21:13 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303719/ [7303720] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_20.04}} time: 00:30:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303720/ [7303721] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:24:03 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303721/ [7303722] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:36 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303722/ [7303723] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_5925} time: 00:24:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303723/ [7303724] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:16:07 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303724/ [7303725] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} time: 00:55:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303725/ [7303726] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:36:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303726/ [7303727] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:30:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303727/ [7303728] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303728/ [7303729] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:21:58 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303729/ [7303730] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:14:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303730/ [7303731] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mix} time: 00:21:42 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303731/ [7303732] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:11:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303732/ [7303733] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:05 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303733/ [7303734] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:20:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303734/ [7303735] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303735/ [7303736] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:13:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303736/ [7303737] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:07:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303737/ [7303738] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:11:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303738/ [7303739] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:11:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303739/ [7303743] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon-stretch} time: 00:20:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303743/ [7303744] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:34:53 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303744/ [7303748] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:11:58 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303748/ [7303753] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:49 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303753/ [7303754] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:18:11 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303754/ [7303755] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:20:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303755/ [7303756] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:48:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303756/ [7303757] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:24 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303757/ [7303758] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:29:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303758/ [7303759] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_api_tests} time: 00:31:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303759/ [7303764] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:34:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303764/ [7303765] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:05 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303765/ [7303766] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:16:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303766/ [7303768] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303768/ [7303769] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} time: 00:33:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303769/ [7303770] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:28:44 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303770/ [7303771] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:59 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303771/ [7303772] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/failover} time: 00:19:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303772/ [7303773] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:23:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303773/ [7303774] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} time: 01:14:06 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303774/ [7303775] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/readwrite} time: 00:17:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303775/ [7303776] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303776/ [7303777] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:07 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303777/ [7303778] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:34:26 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303778/ [7303779] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} time: 00:23:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303779/ [7303780] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:29:47 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303780/ [7303781] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:42 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303781/ [7303782] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:22 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303782/ [7303783] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:34 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303783/ [7303784] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:20:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303784/ [7303785] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303785/ [7303786] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:20:41 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303786/ [7303787] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:13:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303787/ [7303788] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303788/ [7303789] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects} time: 00:21:07 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303789/ [7303790] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:40:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303790/ [7303791] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:54:46 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303791/ [7303792] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:11:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303792/ [7303794] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:18:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303794/ [7303796] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:18:47 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303796/ [7303797] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/repair_test} time: 00:19:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303797/ [7303798] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:40:29 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303798/ [7303799] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:21:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303799/ [7303800] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:29:35 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303800/ [7303801] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:16:07 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303801/ [7303802] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:51 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303802/ [7303805] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 02:56:51 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303805/ [7303808] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_20.04}} time: 00:08:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303808/ [7303809] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303809/ [7303811] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_rgw_multisite} time: 00:19:06 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303811/ [7303812] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:11:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303812/ [7303813] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:28:26 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303813/ [7303814] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:17:28 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303814/ [7303815] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:21 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303815/ [7303816] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303816/ [7303817] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:38 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303817/ [7303820] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:35 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303820/ [7303822] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:24:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303822/ [7303825] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:20:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303825/ [7303827] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits} time: 00:45:19 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303827/ [7303828] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews} time: 00:08:30 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303828/ [7303829] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/lockdep} time: 00:28:28 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303829/ [7303831] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:28:32 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303831/ [7303832] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:13:52 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303832/ [7303833] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 01:39:27 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303833/ [7303834] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:12:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303834/ [7303835] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 02:42:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303835/ [7303836] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303836/ [7303837] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303837/ [7303838] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:23:28 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303838/ [7303839] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:26:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303839/ [7303840] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} time: 00:12:49 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303840/ [7303841] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303841/ [7303842] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:23:18 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303842/ [7303843] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:20 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303843/ [7303844] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303844/ [7303845] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:55:33 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303845/ [7303846] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303846/ [7303847] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:36:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303847/ [7303848] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_8}} time: 00:10:53 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303848/ [7303849] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:17 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303849/ [7303850] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:23 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303850/ [7303851] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:26:31 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303851/ [7303852] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:50 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303852/ [7303853] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:13:03 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303853/ [7303855] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/per_module_finisher_stats} time: 00:16:56 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303855/ [7303856] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:12:13 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303856/ [7303857] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303857/ [7303858] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} time: 00:21:34 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303858/ [7303859] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:34:19 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303859/ [7303860] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/snaps-few-objects} time: 00:35:21 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303860/ [7303861] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:31:21 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303861/ [7303862] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/valgrind} time: 01:55:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303862/ [7303863] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:49 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303863/ [7303864] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:11:02 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303864/ [7303865] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 02:55:05 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303865/ [7303866] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:41 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303866/ [7303867] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:30:00 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303867/ [7303868] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:43 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303868/ [7303869] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:37 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303869/ [7303871] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:04 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303871/ [7303872] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} time: 00:10:43 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303872/ [7303873] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:55 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303873/ [7303874] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:15:54 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303874/ [7303875] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:32:12 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303875/ [7303876] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:04 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303876/ [7303877] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:26:45 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303877/ [7303878] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:10 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303878/ [7303879] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:28:01 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303879/ [7303880] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:09 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303880/ [7303881] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:57 info: http://pulpito.ceph.com/yingxin-2023-06-14_08:25:12-rados-wip-chunmei-ceph-distro-default-smithi/7303881/