2022-12-28T20:40:59.342 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-28T20:40:59.344 INFO:teuthology.results:subset: '111/120000' 2022-12-28T20:40:59.345 INFO:teuthology.results:seed: '2201' 2022-12-29T00:24:26.442 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-29T00:24:26.473 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-12-29T08:58:24.891 INFO:teuthology.results:Tests finished! gathering results... 2022-12-29T08:59:55.853 INFO:scrape:Found 426 jobs 2022-12-29T09:00:11.849 INFO:scrape:Found 11 distinct failure reasons 2022-12-29T09:00:11.850 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2022-12-29T09:00:11.888 INFO:scrape:1 jobs: ['7128793'] 2022-12-29T09:00:11.889 INFO:scrape:suites: ['rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}'] 2022-12-29T09:00:11.890 INFO:scrape: 2022-12-29T09:00:11.891 INFO:scrape:Failure: Command failed (workunit test cephadm/test_repos.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' 2022-12-29T09:00:11.892 INFO:scrape:4 jobs: ['7128634', '7128740', '7128522', '7128845'] 2022-12-29T09:00:11.893 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm_repos}'] 2022-12-29T09:00:11.894 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm_repos}'] 2022-12-29T09:00:11.895 INFO:scrape: 2022-12-29T09:00:11.896 INFO:scrape:Failure: Command failed on smithi110 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdc62526-86fc-11ed-97ac-001a4aab830c -- ceph mon dump -f json' 2022-12-29T09:00:11.897 INFO:scrape:4 jobs: ['7128681', '7128676', '7128612', '7128761'] 2022-12-29T09:00:11.898 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start'] 2022-12-29T09:00:11.898 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/repave-all}', '2-ops/rm-zap-wait}', '2-services/basic', '2-services/nfs-ingress', '3-final}', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8'] 2022-12-29T09:00:11.899 INFO:scrape: 2022-12-29T09:00:11.900 INFO:scrape:Failure: unexpected end of data 2022-12-29T09:00:11.916 INFO:scrape:1 jobs: ['7128589'] 2022-12-29T09:00:11.916 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton/{all/radostool', 'supported-random-distro$/{rhel_8}}'] 2022-12-29T09:00:11.917 INFO:scrape: 2022-12-29T09:00:11.918 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2022-12-29T09:00:11.919 INFO:scrape:1 jobs: ['7128875'] 2022-12-29T09:00:11.920 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2022-12-29T09:00:11.921 INFO:scrape: 2022-12-29T09:00:11.921 INFO:scrape:Failure: {'smithi112.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:06.714933', 'end': '2022-12-28 22:15:45.670569', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-28 22:15:38.955636', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.873521', 'end': '2022-12-28 22:15:45.715309', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-28 22:15:37.841788', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} 2022-12-29T09:00:11.922 INFO:scrape:2 jobs: ['7128689', '7128902'] 2022-12-29T09:00:11.923 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2022-12-29T09:00:11.935 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2022-12-29T09:00:11.935 INFO:scrape: 2022-12-29T09:00:11.936 INFO:scrape:Failure: Command failed on smithi040 with status 1: 'sudo kubeadm init --node-name smithi040 --token abcdef.nrpf7nd53fdcuy8u --pod-network-cidr 10.249.56.0/21' 2022-12-29T09:00:11.937 INFO:scrape:4 jobs: ['7128548', '7128816', '7128683', '7128946'] 2022-12-29T09:00:11.938 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '3-final', 'k8s/1.21', 'net/calico'] 2022-12-29T09:00:11.939 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.6.2}', 'rook/master}'] 2022-12-29T09:00:11.940 INFO:scrape: 2022-12-29T09:00:11.940 INFO:scrape:Failure: Command failed (workunit test post-file.sh) on smithi002 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' 2022-12-29T09:00:11.941 INFO:scrape:2 jobs: ['7128628', '7128841'] 2022-12-29T09:00:11.942 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{centos_8}}'] 2022-12-29T09:00:11.975 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{centos_8}}'] 2022-12-29T09:00:11.976 INFO:scrape: 2022-12-29T09:00:11.977 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2022-12-29T09:00:11.980 INFO:scrape:4 jobs: ['7128930', '7128614', '7128718', '7128828'] 2022-12-29T09:00:11.981 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-12-29T09:00:11.981 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-12-29T09:00:11.982 INFO:scrape: 2022-12-29T09:00:11.983 INFO:scrape:Failure: Command failed on smithi110 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 473efe2e-86ff-11ed-97ac-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-12-29T09:00:11.983 INFO:scrape:1 jobs: ['7128715'] 2022-12-29T09:00:11.984 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2022-12-29T09:00:11.985 INFO:scrape: 2022-12-29T09:00:11.985 INFO:scrape:Dead: b'2022-12-29T08:49:48.021 DEBUG:teuthology.exit:Finished running handlers' 2022-12-29T09:00:11.986 INFO:scrape:1 jobs: ['7128525'] 2022-12-29T09:00:12.043 INFO:scrape:suites: ['rados/objectstore/{backends/objectstore', 'supported-random-distro$/{ubuntu_latest}}'] 2022-12-29T09:00:12.044 INFO:scrape: 2022-12-29T09:00:12.170 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/ failed: 22 dead: 3 running: 0 waiting: 0 queued: 0 passed: 401 Fail ================================================================= [7128522] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:09:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128522/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128522/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9df01fffa3f642ae8762342bde3e6b1d Command failed (workunit test cephadm/test_repos.sh) on smithi067 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7128548] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:08:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128548/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128548/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58012516174b424a8c3c9c28570ceaf1 Command failed on smithi040 with status 1: 'sudo kubeadm init --node-name smithi040 --token abcdef.nrpf7nd53fdcuy8u --pod-network-cidr 10.249.56.0/21' [7128589] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:24:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128589/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128589/ unexpected end of data [7128612] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:13:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128612/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128612/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc345ac3e9244746891145c9037c4527 Command failed on smithi043 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7ea10d3c-86f8-11ed-97ac-001a4aab830c -- ceph osd stat -f json' [7128614] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128614/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128614/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d77b603757b4458284b7a6093b4b4cd7 Command failed (workunit test cephadm/test_cephadm.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7128628] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128628/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128628/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69860e4b627d4c02b1ea4ed0f2a8770d Command failed (workunit test post-file.sh) on smithi002 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7128634] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:10:11 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128634/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128634/ sentry: https://sentry.ceph.com/organizations/ceph/?query=946d234a8fd14360af516d721b8d1a48 Command failed (workunit test cephadm/test_repos.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7128676] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128676/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128676/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8ddc417855b4dbb98699e66fd47831e Command failed on smithi157 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b4d559b8-86fc-11ed-97ac-001a4aab830c -- ceph mon dump -f json' [7128681] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128681/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128681/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1272f7affcd24f58a3c95544b69a475b Command failed on smithi110 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdc62526-86fc-11ed-97ac-001a4aab830c -- ceph mon dump -f json' [7128683] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:08:35 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128683/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128683/ sentry: https://sentry.ceph.com/organizations/ceph/?query=355b5449f3e54349852e5af41fd2fb78 Command failed on smithi072 with status 1: 'sudo kubeadm init --node-name smithi072 --token abcdef.v61nxt43nvohqs11 --pod-network-cidr 10.250.56.0/21' [7128715] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128715/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128715/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6458b28e2e2f4a029a1d62511118b939 Command failed on smithi110 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 473efe2e-86ff-11ed-97ac-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7128718] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:13:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128718/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128718/ sentry: https://sentry.ceph.com/organizations/ceph/?query=556c93f6667e447990d7c404d4b56979 Command failed (workunit test cephadm/test_cephadm.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7128740] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:09:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128740/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128740/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cf291558babe4fa19f18f6908f26c7b8 Command failed (workunit test cephadm/test_repos.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7128761] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128761/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128761/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f4b4ba98fee4f6080ea793808c43baa Command failed on smithi139 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:0a461c9fb3063e8664ae45ade45df7828631680d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca08ca12-8702-11ed-97ac-001a4aab830c -- ceph osd stat -f json' [7128793] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} ----------------------------------------------------------------- time: 00:23:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128793/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128793/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f75c50b1a8524af1be3285b62106c209 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7128816] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:08:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128816/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128816/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8d34fc1f8a5e4cb699b706fb090b4f11 Command failed on smithi060 with status 1: 'sudo kubeadm init --node-name smithi060 --token abcdef.1qifwg40tqi2a24y --pod-network-cidr 10.249.216.0/21' [7128828] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:12:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128828/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128828/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5b25168061604595a8a71a635fcb222e Command failed (workunit test cephadm/test_cephadm.sh) on smithi088 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7128841] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:11:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128841/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128841/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1a9d9d10a352489eb2d0d8a4f4c59d45 Command failed (workunit test post-file.sh) on smithi078 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7128845] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128845/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128845/ sentry: https://sentry.ceph.com/organizations/ceph/?query=26ae211120a34c6d9cc46c58dc1a9549 Command failed (workunit test cephadm/test_repos.sh) on smithi102 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [7128875] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 01:13:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128875/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128875/ sentry: https://sentry.ceph.com/organizations/ceph/?query=45d5844c759d4522a7da3443ea1f231c Command failed (workunit test rados/test.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7128930] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:12:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128930/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128930/ sentry: https://sentry.ceph.com/organizations/ceph/?query=04f9a91e07234a389d4855d25a0476a9 Command failed (workunit test cephadm/test_cephadm.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a461c9fb3063e8664ae45ade45df7828631680d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7128946] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:08:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128946/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128946/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e17e33332c34468b2c3d8056aaa861b Command failed on smithi052 with status 1: 'sudo kubeadm init --node-name smithi052 --token abcdef.8vdhhstxxc316fft --pod-network-cidr 10.249.152.0/21' Dead ================================================================= [7128525] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128525/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128525/ hit max job timeout [7128689] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:05:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128689/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128689/ sentry: https://sentry.ceph.com/organizations/ceph/?query=63e5edeed413467ba0f6c3caeba632a9 {'smithi112.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:06.714933', 'end': '2022-12-28 22:15:45.670569', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-28 22:15:38.955636', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.873521', 'end': '2022-12-28 22:15:45.715309', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-28 22:15:37.841788', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} [7128902] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:04:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128902/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d393e9d6ce674a039ce215a26bb0e0d2 {'smithi120.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.142812', 'end': '2022-12-29 00:06:05.618474', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-29 00:06:01.475662', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi155.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.396969', 'end': '2022-12-29 00:06:05.748425', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-29 00:06:01.351456', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2511 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2511 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} Pass ================================================================= [7128523] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:21:57 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128523/ [7128524] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:49:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128524/ [7128526] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:20:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128526/ [7128527] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128527/ [7128528] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128528/ [7128529] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:23:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128529/ [7128530] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128530/ [7128531] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128531/ [7128532] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:32:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128532/ [7128533] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:32:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128533/ [7128534] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128534/ [7128535] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:44:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128535/ [7128536] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:28:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128536/ [7128537] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:13:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128537/ [7128538] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:34:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128538/ [7128539] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:26:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128539/ [7128540] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:29:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128540/ [7128541] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:08:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128541/ [7128542] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128542/ [7128543] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128543/ [7128544] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:40:23 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128544/ [7128545] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128545/ [7128546] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:20:35 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128546/ [7128547] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:23:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128547/ [7128549] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128549/ [7128550] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:44:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128550/ [7128551] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:42:33 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128551/ [7128552] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:18:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128552/ [7128553] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128553/ [7128554] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:36:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128554/ [7128555] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128555/ [7128556] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128556/ [7128557] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:20:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128557/ [7128558] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128558/ [7128559] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:33 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128559/ [7128560] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128560/ [7128561] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128561/ [7128562] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128562/ [7128563] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:54:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128563/ [7128564] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:35:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128564/ [7128565] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:25:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128565/ [7128566] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} time: 00:29:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128566/ [7128567] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:12:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128567/ [7128568] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128568/ [7128569] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128569/ [7128570] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128570/ [7128571] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128571/ [7128572] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:21:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128572/ [7128573] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{filestore-xfs} supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:28:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128573/ [7128574] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:12:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128574/ [7128575] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:16:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128575/ [7128576] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128576/ [7128577] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128577/ [7128578] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} time: 03:12:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128578/ [7128579] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:23:57 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128579/ [7128580] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:25:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128580/ [7128581] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128581/ [7128582] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128582/ [7128583] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:08:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128583/ [7128584] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:25:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128584/ [7128585] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:32:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128585/ [7128586] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:15:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128586/ [7128587] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128587/ [7128588] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr orchestrator_cli} time: 00:17:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128588/ [7128590] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:18:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128590/ [7128591] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128591/ [7128592] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:45 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128592/ [7128593] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:08:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128593/ [7128594] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128594/ [7128595] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128595/ [7128596] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:26:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128596/ [7128597] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128597/ [7128598] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:35:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128598/ [7128599] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:39:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128599/ [7128600] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:30:52 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128600/ [7128601] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:29:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128601/ [7128602] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:27:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128602/ [7128603] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128603/ [7128604] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:38:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128604/ [7128605] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:16:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128605/ [7128606] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128606/ [7128607] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128607/ [7128608] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:28:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128608/ [7128609] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128609/ [7128610] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:34:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128610/ [7128611] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128611/ [7128613] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128613/ [7128615] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:07:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128615/ [7128616] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:19:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128616/ [7128617] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:48:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128617/ [7128618] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:20:36 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128618/ [7128619] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128619/ [7128620] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:35:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128620/ [7128621] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128621/ [7128622] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128622/ [7128623] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:31:38 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128623/ [7128624] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:17:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128624/ [7128625] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 01:10:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128625/ [7128626] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:16:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128626/ [7128627] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128627/ [7128629] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:53:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128629/ [7128630] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:13:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128630/ [7128631] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:30:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128631/ [7128632] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:32:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128632/ [7128633] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128633/ [7128635] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:19:25 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128635/ [7128636] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:19:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128636/ [7128637] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:46:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128637/ [7128638] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128638/ [7128639] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128639/ [7128640] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:10:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128640/ [7128641] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:31:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128641/ [7128642] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:30:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128642/ [7128643] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:38 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128643/ [7128644] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:21 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128644/ [7128645] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:36 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128645/ [7128646] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:58:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128646/ [7128647] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128647/ [7128648] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128648/ [7128649] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:29:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128649/ [7128650] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:12:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128650/ [7128651] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128651/ [7128652] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:25:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128652/ [7128653] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:28:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128653/ [7128654] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:23:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128654/ [7128655] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:37:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128655/ [7128656] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:07:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128656/ [7128657] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128657/ [7128658] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128658/ [7128659] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:19:11 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128659/ [7128660] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:18:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128660/ [7128661] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128661/ [7128662] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:35:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128662/ [7128663] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:19:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128663/ [7128664] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:15:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128664/ [7128665] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:37:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128665/ [7128666] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128666/ [7128667] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:42:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128667/ [7128668] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128668/ [7128669] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:09:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128669/ [7128670] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:34:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128670/ [7128671] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:17:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128671/ [7128672] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:12:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128672/ [7128673] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128673/ [7128674] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 01:22:57 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128674/ [7128675] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:23 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128675/ [7128677] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128677/ [7128678] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:26:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128678/ [7128679] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:36:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128679/ [7128680] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128680/ [7128682] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} time: 00:30:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128682/ [7128684] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:18:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128684/ [7128685] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:42:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128685/ [7128686] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:55:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128686/ [7128687] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:16:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128687/ [7128688] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128688/ [7128690] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 01:16:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128690/ [7128691] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128691/ [7128692] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128692/ [7128693] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:20:21 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128693/ [7128694] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128694/ [7128695] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128695/ [7128696] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128696/ [7128697] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128697/ [7128698] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:39:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128698/ [7128699] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128699/ [7128700] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} time: 00:31:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128700/ [7128701] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:17:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128701/ [7128702] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:33:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128702/ [7128703] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:27:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128703/ [7128704] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:30:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128704/ [7128705] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128705/ [7128706] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:34:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128706/ [7128707] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:09:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128707/ [7128708] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128708/ [7128709] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:20:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128709/ [7128710] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128710/ [7128711] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:18:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128711/ [7128712] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:36:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128712/ [7128713] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128713/ [7128714] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128714/ [7128716] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:12:23 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128716/ [7128717] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:43:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128717/ [7128719] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:38:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128719/ [7128720] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:26:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128720/ [7128721] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:38:00 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128721/ [7128722] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:20:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128722/ [7128723] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:21:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128723/ [7128724] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128724/ [7128725] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128725/ [7128726] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:25:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128726/ [7128727] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:34:33 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128727/ [7128728] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:12:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128728/ [7128729] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:38:11 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128729/ [7128730] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:35:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128730/ [7128731] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128731/ [7128732] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128732/ [7128733] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:23:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128733/ [7128734] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:37:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128734/ [7128735] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:39:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128735/ [7128736] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128736/ [7128737] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128737/ [7128738] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:28:57 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128738/ [7128739] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:21 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128739/ [7128741] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128741/ [7128742] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:18:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128742/ [7128743] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:19:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128743/ [7128744] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:17:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128744/ [7128745] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128745/ [7128746] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128746/ [7128747] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:26:21 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128747/ [7128748] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:21:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128748/ [7128749] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:36:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128749/ [7128750] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128750/ [7128751] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:14:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128751/ [7128752] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128752/ [7128753] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:47:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128753/ [7128754] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128754/ [7128755] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:47:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128755/ [7128756] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:31:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128756/ [7128757] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128757/ [7128758] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128758/ [7128759] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:22:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128759/ [7128760] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128760/ [7128762] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:43:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128762/ [7128763] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 01:00:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128763/ [7128764] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:30:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128764/ [7128765] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:31:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128765/ [7128766] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:09:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128766/ [7128767] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:30:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128767/ [7128768] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:26:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128768/ [7128769] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:17:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128769/ [7128770] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128770/ [7128771] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:22:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128771/ [7128772] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:37:11 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128772/ [7128773] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:35 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128773/ [7128774] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128774/ [7128775] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128775/ [7128776] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128776/ [7128777] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128777/ [7128778] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:31:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128778/ [7128779] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:35:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128779/ [7128780] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:13:18 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128780/ [7128781] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:24:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128781/ [7128782] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:07:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128782/ [7128783] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} time: 00:28:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128783/ [7128784] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:34:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128784/ [7128785] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128785/ [7128786] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:23:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128786/ [7128787] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128787/ [7128788] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:51:17 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128788/ [7128789] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:57:36 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128789/ [7128790] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:22:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128790/ [7128791] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:15:42 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128791/ [7128792] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128792/ [7128794] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:17:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128794/ [7128795] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:30:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128795/ [7128796] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:40 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128796/ [7128797] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128797/ [7128798] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:33:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128798/ [7128799] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:05:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128799/ [7128800] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/readwrite} time: 00:15:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128800/ [7128801] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:18:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128801/ [7128802] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128802/ [7128803] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128803/ [7128804] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} time: 00:20:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128804/ [7128805] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 01:12:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128805/ [7128806] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:19:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128806/ [7128807] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:17:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128807/ [7128808] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:04:36 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128808/ [7128809] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128809/ [7128810] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:32:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128810/ [7128811] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:36:32 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128811/ [7128812] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:07:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128812/ [7128813] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:10:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128813/ [7128814] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128814/ [7128815] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:34:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128815/ [7128817] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:41:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128817/ [7128818] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128818/ [7128819] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 00:51:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128819/ [7128820] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128820/ [7128821] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:28:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128821/ [7128822] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128822/ [7128823] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:56:35 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128823/ [7128824] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:18:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128824/ [7128825] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128825/ [7128826] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:25 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128826/ [7128827] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128827/ [7128829] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:18:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128829/ [7128830] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:16:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128830/ [7128831] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:33:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128831/ [7128832] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128832/ [7128833] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128833/ [7128834] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:14:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128834/ [7128835] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128835/ [7128836] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/prometheus} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128836/ [7128837] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:12:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128837/ [7128838] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:57:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128838/ [7128839] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128839/ [7128840] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128840/ [7128842] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:41:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128842/ [7128843] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:32:36 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128843/ [7128844] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128844/ [7128846] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:16:53 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128846/ [7128847] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128847/ [7128848] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128848/ [7128849] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:19:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128849/ [7128850] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128850/ [7128851] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128851/ [7128852] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:25:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128852/ [7128853] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:46:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128853/ [7128854] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128854/ [7128855] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:12:02 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128855/ [7128856] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128856/ [7128857] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:39:23 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128857/ [7128858] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:53:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128858/ [7128859] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128859/ [7128860] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:23:45 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128860/ [7128861] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128861/ [7128862] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:28:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128862/ [7128863] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:11:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128863/ [7128864] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128864/ [7128865] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:18:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128865/ [7128866] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:22:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128866/ [7128867] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:16:01 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128867/ [7128868] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:23:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128868/ [7128869] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128869/ [7128870] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 01:05:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128870/ [7128871] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:10 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128871/ [7128872] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128872/ [7128873] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128873/ [7128874] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:31:25 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128874/ [7128876] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128876/ [7128877] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:40:16 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128877/ [7128878] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/workunits} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128878/ [7128879] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} time: 00:33:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128879/ [7128880] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:12:38 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128880/ [7128881] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:37:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128881/ [7128882] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:22:09 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128882/ [7128883] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:20:15 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128883/ [7128884] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:17:09 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128884/ [7128885] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:06 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128885/ [7128886] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:10:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128886/ [7128887] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:36:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128887/ [7128888] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128888/ [7128889] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:38 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128889/ [7128890] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128890/ [7128891] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:24:51 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128891/ [7128892] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128892/ [7128893] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects} time: 00:25:22 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128893/ [7128894] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} time: 00:28:57 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128894/ [7128895] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:33:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128895/ [7128896] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:19:48 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128896/ [7128897] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:36:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128897/ [7128898] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:28:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128898/ [7128899] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:17:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128899/ [7128900] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128900/ [7128901] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128901/ [7128903] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:33 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128903/ [7128904] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:26 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128904/ [7128905] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:50 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128905/ [7128906] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:18:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128906/ [7128907] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128907/ [7128908] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128908/ [7128909] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:09:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128909/ [7128910] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:09:17 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128910/ [7128911] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:52:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128911/ [7128912] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:43 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128912/ [7128913] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:16:25 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128913/ [7128914] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} time: 00:47:58 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128914/ [7128915] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:20 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128915/ [7128916] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128916/ [7128917] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:27:58 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128917/ [7128918] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:20:37 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128918/ [7128919] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:12:27 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128919/ [7128920] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128920/ [7128921] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:32:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128921/ [7128922] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128922/ [7128923] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:26:49 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128923/ [7128924] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:20:13 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128924/ [7128925] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:52:07 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128925/ [7128926] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128926/ [7128927] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:28:19 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128927/ [7128928] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128928/ [7128929] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:28:14 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128929/ [7128931] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:19:03 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128931/ [7128932] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:46:58 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128932/ [7128933] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:21:59 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128933/ [7128934] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:46 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128934/ [7128935] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:29:04 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128935/ [7128936] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:34:43 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128936/ [7128937] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:24:28 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128937/ [7128938] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128938/ [7128939] rados/standalone/{supported-random-distro$/{centos_8} workloads/scrub} time: 01:44:12 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128939/ [7128940] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:34:39 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128940/ [7128941] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:20:25 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128941/ [7128942] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128942/ [7128943] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:31 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128943/ [7128944] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:29 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128944/ [7128945] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:34:43 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128945/ [7128947] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:37:56 info: http://pulpito.ceph.com/yuriw-2022-12-28_16:54:36-rados-pacific_16.2.11_RC3.5-distro-default-smithi/7128947/