2022-12-21T19:59:51.361 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-21T19:59:51.361 INFO:teuthology.results:subset: '111/120000' 2022-12-21T19:59:51.361 INFO:teuthology.results:seed: '3193' 2022-12-22T01:16:08.808 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-22T01:16:09.539 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-12-22T03:52:26.377 INFO:teuthology.results:Tests finished! gathering results... 2022-12-22T03:52:57.635 INFO:scrape:Found 426 jobs 2022-12-22T03:52:58.216 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124196/teuthology.log 2022-12-22T03:53:05.316 INFO:scrape:Found 11 distinct failure reasons 2022-12-22T03:53:05.316 INFO:scrape:Failure: {'smithi003.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.170067', 'end': '2022-12-22 00:55:52.206990', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-22 00:55:45.036923', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi023.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.439198', 'end': '2022-12-22 00:55:52.377938', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-22 00:55:44.938740', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} 2022-12-22T03:53:05.316 INFO:scrape:2 jobs: ['7124397', '7124184'] 2022-12-22T03:53:05.317 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2022-12-22T03:53:05.317 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2022-12-22T03:53:05.317 INFO:scrape: 2022-12-22T03:53:05.317 INFO:scrape:Failure: Command failed on smithi023 with status 1: 'sudo kubeadm init --node-name smithi023 --token abcdef.k4ldri2ayajqsvjv --pod-network-cidr 10.248.176.0/21' 2022-12-22T03:53:05.317 INFO:scrape:4 jobs: ['7124178', '7124311', '7124043', '7124441'] 2022-12-22T03:53:05.317 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '3-final', 'k8s/1.21', 'net/calico'] 2022-12-22T03:53:05.317 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.6.2}', 'rook/master}'] 2022-12-22T03:53:05.317 INFO:scrape: 2022-12-22T03:53:05.318 INFO:scrape:Failure: Command failed (workunit test post-file.sh) on smithi157 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' 2022-12-22T03:53:05.318 INFO:scrape:2 jobs: ['7124123', '7124336'] 2022-12-22T03:53:05.318 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/ceph-post-file'] 2022-12-22T03:53:05.318 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}'] 2022-12-22T03:53:05.318 INFO:scrape: 2022-12-22T03:53:05.318 INFO:scrape:Failure: Command failed on smithi083 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:22be1b8fdf3b0a37691ef6abcf04f7c57402534b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 209da85e-818e-11ed-97a8-001a4aab830c -- ceph osd stat -f json' 2022-12-22T03:53:05.318 INFO:scrape:1 jobs: ['7124343'] 2022-12-22T03:53:05.319 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/repave-all}', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0'] 2022-12-22T03:53:05.319 INFO:scrape: 2022-12-22T03:53:05.319 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2022-12-22T03:53:05.319 INFO:scrape:4 jobs: ['7124213', '7124323', '7124109', '7124425'] 2022-12-22T03:53:05.319 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-12-22T03:53:05.319 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2022-12-22T03:53:05.319 INFO:scrape: 2022-12-22T03:53:05.320 INFO:scrape:Dead: None 2022-12-22T03:53:05.320 INFO:scrape:8 jobs: ['7124196', '7124253', '7124308', '7124128', '7124080', '7124246', '7124057', '7124380'] 2022-12-22T03:53:05.320 INFO:scrape: 2022-12-22T03:53:05.320 INFO:scrape:Failure: "2022-12-22T00:35:18.406588+0000 mon.b (mon.1) 211 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log 2022-12-22T03:53:05.320 INFO:scrape:2 jobs: ['7124305', '7124113'] 2022-12-22T03:53:05.320 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'openstack}', 'rados', 'supported-random-distro$/{centos_8}', 'thrashosds-health'] 2022-12-22T03:53:05.320 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{default}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'fast/fast', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_8}', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-small-objects-many-deletes}', 'workloads/small-objects}'] 2022-12-22T03:53:05.320 INFO:scrape: 2022-12-22T03:53:05.321 INFO:scrape:Failure: Command failed on smithi046 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5eba5420-8189-11ed-97a8-001a4aab830c -e sha1=22be1b8fdf3b0a37691ef6abcf04f7c57402534b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2022-12-22T03:53:05.321 INFO:scrape:1 jobs: ['7124291'] 2022-12-22T03:53:05.321 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/no', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4'] 2022-12-22T03:53:05.321 INFO:scrape: 2022-12-22T03:53:05.321 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2022-12-22T03:53:05.321 INFO:scrape:1 jobs: ['7124075'] 2022-12-22T03:53:05.321 INFO:scrape:suites: ['rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}'] 2022-12-22T03:53:05.321 INFO:scrape: 2022-12-22T03:53:05.321 INFO:scrape:Failure: Command failed on smithi031 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' 2022-12-22T03:53:05.322 INFO:scrape:2 jobs: ['7124068', '7124073'] 2022-12-22T03:53:05.322 INFO:scrape:suites intersection: ['mon_election/classic'] 2022-12-22T03:53:05.322 INFO:scrape:suites union: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-pacific', '8-workload/{rbd-python', 'bluestore-bitmap', 'clusters/{2-node-mgr}', 'debug/mgr', 'mon_election/classic', 'rados/dashboard/{centos_8.stream_container_tools', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'random-objectstore$/{bluestore-comp-snappy}', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/dashboard}', 'thrashosds-health', 'ubuntu_18.04}'] 2022-12-22T03:53:05.322 INFO:scrape: 2022-12-22T03:53:05.322 INFO:scrape:Failure: Command failed on smithi109 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:22be1b8fdf3b0a37691ef6abcf04f7c57402534b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 925142fc-8180-11ed-97a8-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-12-22T03:53:05.322 INFO:scrape:1 jobs: ['7124210'] 2022-12-22T03:53:05.322 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2022-12-22T03:53:05.322 INFO:scrape: 2022-12-22T03:53:05.336 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/ failed: 18 dead: 10 running: 0 waiting: 0 queued: 0 passed: 398 Fail ================================================================= [7124043] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:07:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124043/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124043/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7b21d67d499d4dcfba5c953207eabc7f Command failed on smithi137 with status 1: 'sudo kubeadm init --node-name smithi137 --token abcdef.4zgf9el512ryvs38 --pod-network-cidr 10.252.64.0/21' [7124068] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/dashboard} ----------------------------------------------------------------- time: 00:04:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124068/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124068/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6d46d4d44cad4837adf5098cf1b25a9a Command failed on smithi031 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' [7124073] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124073/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124073/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0deefb243b12470481ca5e367789cf06 Command failed on smithi074 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' [7124075] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} ----------------------------------------------------------------- time: 00:22:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124075/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124075/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2a2254dee65242c8b342fe3d66382653 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7124109] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:11:37 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124109/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124109/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3972514f196a49e09fb40b06792f6cfc Command failed (workunit test cephadm/test_cephadm.sh) on smithi061 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7124113] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:27:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124113/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124113/ "2022-12-21T21:34:36.522093+0000 mon.a (mon.0) 1353 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [7124123] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:07:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124123/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124123/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3634e80795584cf1b8dd382e7b9773cf Command failed (workunit test post-file.sh) on smithi157 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7124178] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:07:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124178/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124178/ sentry: https://sentry.ceph.com/organizations/ceph/?query=011a4afb8fd448b195e7dceb5ba7322f Command failed on smithi023 with status 1: 'sudo kubeadm init --node-name smithi023 --token abcdef.k4ldri2ayajqsvjv --pod-network-cidr 10.248.176.0/21' [7124210] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:15:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124210/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124210/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da0899748a4549cdb8619e4b13b16fd5 Command failed on smithi109 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:22be1b8fdf3b0a37691ef6abcf04f7c57402534b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 925142fc-8180-11ed-97a8-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7124213] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124213/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124213/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e1001d807404368b49efcb6cf2fc5ec Command failed (workunit test cephadm/test_cephadm.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7124291] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:23:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124291/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124291/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6bbcfbf174c49eaa74d25d75d7baf75 Command failed on smithi046 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5eba5420-8189-11ed-97a8-001a4aab830c -e sha1=22be1b8fdf3b0a37691ef6abcf04f7c57402534b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7124305] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:28:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124305/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124305/ "2022-12-22T00:35:18.406588+0000 mon.b (mon.1) 211 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log [7124311] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:08:17 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124311/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124311/ sentry: https://sentry.ceph.com/organizations/ceph/?query=afea3bbf5c2f45b797aa83ba515318f0 Command failed on smithi007 with status 1: 'sudo kubeadm init --node-name smithi007 --token abcdef.ptqd9rnrhbft1tc9 --pod-network-cidr 10.248.48.0/21' [7124323] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:11:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124323/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124323/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e654bfc7d07646f3b4c65fcd982b24ea Command failed (workunit test cephadm/test_cephadm.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7124336] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:06:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124336/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124336/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4184b318f14147398ba36438c2790ec2 Command failed (workunit test post-file.sh) on smithi032 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7124343] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:09:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124343/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124343/ sentry: https://sentry.ceph.com/organizations/ceph/?query=42e860ac3bef45a2b2722e9758d4eaee Command failed on smithi083 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:22be1b8fdf3b0a37691ef6abcf04f7c57402534b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 209da85e-818e-11ed-97a8-001a4aab830c -- ceph osd stat -f json' [7124425] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:11:17 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124425/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124425/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bb97258a072d425e8b6d37c3263d12d9 Command failed (workunit test cephadm/test_cephadm.sh) on smithi088 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=22be1b8fdf3b0a37691ef6abcf04f7c57402534b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7124441] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:07:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124441/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124441/ sentry: https://sentry.ceph.com/organizations/ceph/?query=997f3c8682eb48c9a3c7ce890b98e926 Command failed on smithi007 with status 1: 'sudo kubeadm init --node-name smithi007 --token abcdef.f9417rffa4tfm2mv --pod-network-cidr 10.248.48.0/21' Dead ================================================================= [7124057] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124057/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124057/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124080] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124080/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124080/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124128] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124128/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124128/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124184] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:03:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124184/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124184/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9d372ce6b34643c49343ee806ac3d879 {'smithi045.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.382893', 'end': '2022-12-21 22:15:16.086489', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-21 22:15:11.703596', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi138.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.419519', 'end': '2022-12-21 22:15:16.141483', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-21 22:15:11.721964', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} [7124196] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124196/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124196/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124246] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124246/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124246/ Error reimaging machines: Command failed on smithi074 with status 4: "sudo sed -i -e 's/smithi074.front.sepia.ceph.com/smithi074/g' /etc/hosts" [7124253] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124253/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124253/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124308] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124308/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124308/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124380] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124380/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124380/ Error reimaging machines: reached maximum tries (180) after waiting for 2700 seconds [7124397] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:04:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124397/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124397/ sentry: https://sentry.ceph.com/organizations/ceph/?query=21b67aa09ad2478894f357b339506f6f {'smithi003.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.170067', 'end': '2022-12-22 00:55:52.206990', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-22 00:55:45.036923', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi023.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:07.439198', 'end': '2022-12-22 00:55:52.377938', 'invocation': {'module_args': {'_raw_params': 'cpan Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-22 00:55:44.938740', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/. cpan/sources/authors/01mailrc.txt.gz'\n.................................... ........................................DONE\nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2504 days old.\n Please check the host you chose as your CPAN mirror for staleness.\n I'll continue but problems seem likely to happen.\x07\n...... ......................................................................DONE\ nFetching with LWP:\nhttp://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '................... .........................................................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", ' Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2504 days old.', ' Please check the host you chose as your CPAN mirror for staleness.', " I'll continue but problems seem likely to happen.\x07", '............................................................ ................DONE', 'Fetching with LWP:', 'http://apt- mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}} Pass ================================================================= [7124017] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:08:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124017/ [7124018] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124018/ [7124019] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:29:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124019/ [7124020] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 05:09:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124020/ [7124021] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:15:23 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124021/ [7124022] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:16:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124022/ [7124023] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124023/ [7124024] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:21:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124024/ [7124025] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:12:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124025/ [7124026] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124026/ [7124027] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:26:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124027/ [7124028] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:32:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124028/ [7124029] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124029/ [7124030] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:41:20 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124030/ [7124031] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:25:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124031/ [7124032] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/prometheus} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124032/ [7124033] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:33:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124033/ [7124034] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:24:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124034/ [7124035] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:27:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124035/ [7124036] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:08:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124036/ [7124037] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124037/ [7124038] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:37 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124038/ [7124039] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:37:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124039/ [7124040] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124040/ [7124041] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:21:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124041/ [7124042] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:21:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124042/ [7124044] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124044/ [7124045] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:29:50 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124045/ [7124046] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:40:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124046/ [7124047] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124047/ [7124048] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:24:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124048/ [7124049] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:32:38 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124049/ [7124050] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124050/ [7124051] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124051/ [7124052] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124052/ [7124053] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124053/ [7124054] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124054/ [7124055] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:23 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124055/ [7124056] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124056/ [7124058] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:51:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124058/ [7124059] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:31:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124059/ [7124060] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:23:37 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124060/ [7124061] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} time: 00:27:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124061/ [7124062] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124062/ [7124063] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:17:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124063/ [7124064] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124064/ [7124065] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124065/ [7124066] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:10:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124066/ [7124067] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:20:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124067/ [7124069] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124069/ [7124070] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124070/ [7124071] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124071/ [7124072] rados/standalone/{supported-random-distro$/{centos_8} workloads/crush} time: 00:14:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124072/ [7124074] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:23:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124074/ [7124076] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124076/ [7124077] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:28:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124077/ [7124078] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/workunits} time: 00:09:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124078/ [7124079] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:24:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124079/ [7124081] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:19:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124081/ [7124082] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124082/ [7124083] rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr orchestrator_cli} time: 00:11:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124083/ [7124084] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:24:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124084/ [7124085] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:26:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124085/ [7124086] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:52 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124086/ [7124087] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:25 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124087/ [7124088] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:07:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124088/ [7124089] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124089/ [7124090] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124090/ [7124091] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:22:29 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124091/ [7124092] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/basic 3-final} time: 00:15:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124092/ [7124093] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124093/ [7124094] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:37:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124094/ [7124095] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124095/ [7124096] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124096/ [7124097] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:26:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124097/ [7124098] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124098/ [7124099] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:35:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124099/ [7124100] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:14:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124100/ [7124101] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124101/ [7124102] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:22 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124102/ [7124103] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:22:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124103/ [7124104] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124104/ [7124105] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:34:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124105/ [7124106] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124106/ [7124107] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124107/ [7124108] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124108/ [7124110] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:07:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124110/ [7124111] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124111/ [7124112] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:46:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124112/ [7124114] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124114/ [7124115] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:36:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124115/ [7124116] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124116/ [7124117] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:10:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124117/ [7124118] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} time: 00:31:52 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124118/ [7124119] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:13:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124119/ [7124120] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:55:55 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124120/ [7124121] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124121/ [7124122] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124122/ [7124124] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:52:56 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124124/ [7124125] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124125/ [7124126] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124126/ [7124127] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:24:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124127/ [7124129] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:08:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124129/ [7124130] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124130/ [7124131] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:17:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124131/ [7124132] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:42:46 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124132/ [7124133] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:16:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124133/ [7124134] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124134/ [7124135] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:10:44 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124135/ [7124136] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:27:44 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124136/ [7124137] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:30:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124137/ [7124138] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124138/ [7124139] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124139/ [7124140] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124140/ [7124141] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:53:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124141/ [7124142] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124142/ [7124143] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124143/ [7124144] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:27:49 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124144/ [7124145] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124145/ [7124146] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124146/ [7124147] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:24:22 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124147/ [7124148] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:27:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124148/ [7124149] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:22:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124149/ [7124150] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:36:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124150/ [7124151] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124151/ [7124152] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124152/ [7124153] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124153/ [7124154] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124154/ [7124155] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:17:05 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124155/ [7124156] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:23 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124156/ [7124157] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:34:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124157/ [7124158] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124158/ [7124159] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:12:19 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124159/ [7124160] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects} time: 00:37:24 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124160/ [7124161] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:18:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124161/ [7124162] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:41:55 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124162/ [7124163] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124163/ [7124164] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:07:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124164/ [7124165] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:30:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124165/ [7124166] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124166/ [7124167] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:08:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124167/ [7124168] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124168/ [7124169] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 01:21:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124169/ [7124170] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124170/ [7124171] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124171/ [7124172] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:09:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124172/ [7124173] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124173/ [7124174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:33:05 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124174/ [7124175] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:07:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124175/ [7124176] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124176/ [7124177] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} time: 00:26:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124177/ [7124179] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:57:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124179/ [7124180] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:38:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124180/ [7124181] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:52:44 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124181/ [7124182] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124182/ [7124183] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124183/ [7124185] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 01:10:50 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124185/ [7124186] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124186/ [7124187] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124187/ [7124188] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:20:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124188/ [7124189] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:17:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124189/ [7124190] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124190/ [7124191] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124191/ [7124192] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124192/ [7124193] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:38:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124193/ [7124194] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124194/ [7124195] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:28:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124195/ [7124197] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:28 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124197/ [7124198] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124198/ [7124199] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:28:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124199/ [7124200] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124200/ [7124201] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:30:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124201/ [7124202] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:49 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124202/ [7124203] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/insights} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124203/ [7124204] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124204/ [7124205] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124205/ [7124206] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:15:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124206/ [7124207] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:33:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124207/ [7124208] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124208/ [7124209] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:21:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124209/ [7124211] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:08:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124211/ [7124212] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124212/ [7124214] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{ubuntu_latest}} time: 02:29:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124214/ [7124215] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:22:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124215/ [7124216] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:35:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124216/ [7124217] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124217/ [7124218] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:18:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124218/ [7124219] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124219/ [7124220] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124220/ [7124221] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:21:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124221/ [7124222] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:24:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124222/ [7124223] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:10:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124223/ [7124224] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:35:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124224/ [7124225] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:33:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124225/ [7124226] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124226/ [7124227] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124227/ [7124228] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:20:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124228/ [7124229] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:36:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124229/ [7124230] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:35:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124230/ [7124231] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:37 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124231/ [7124232] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:40:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124232/ [7124233] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124233/ [7124234] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124234/ [7124235] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:08:50 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124235/ [7124236] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:10:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124236/ [7124237] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:16:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124237/ [7124238] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124238/ [7124239] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:14:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124239/ [7124240] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:08:09 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124240/ [7124241] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124241/ [7124242] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:25:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124242/ [7124243] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:22:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124243/ [7124244] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124244/ [7124245] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124245/ [7124247] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124247/ [7124248] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:46:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124248/ [7124249] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124249/ [7124250] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 02:03:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124250/ [7124251] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:32:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124251/ [7124252] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124252/ [7124254] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:22:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124254/ [7124255] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124255/ [7124256] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:17:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124256/ [7124257] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:43:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124257/ [7124258] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:51:18 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124258/ [7124259] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} time: 02:28:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124259/ [7124260] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:33:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124260/ [7124261] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:09:46 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124261/ [7124262] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124262/ [7124263] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:23:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124263/ [7124264] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 00:15:29 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124264/ [7124265] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:11:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124265/ [7124266] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:20:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124266/ [7124267] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:30:25 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124267/ [7124268] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124268/ [7124269] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124269/ [7124270] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124270/ [7124271] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124271/ [7124272] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:28 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124272/ [7124273] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124273/ [7124274] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:35:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124274/ [7124275] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:10:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124275/ [7124276] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:24:38 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124276/ [7124277] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:08:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124277/ [7124278] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} time: 00:27:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124278/ [7124279] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:33:23 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124279/ [7124280] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:18:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124280/ [7124281] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/dashboard} time: 01:04:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124281/ [7124282] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:17:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124282/ [7124283] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:42:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124283/ [7124284] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:52:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124284/ [7124285] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:18:56 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124285/ [7124286] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:07:17 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124286/ [7124287] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124287/ [7124288] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} time: 00:24:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124288/ [7124289] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/progress} time: 00:16:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124289/ [7124290] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:28:28 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124290/ [7124292] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:07:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124292/ [7124293] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:30:26 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124293/ [7124294] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:07:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124294/ [7124295] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:14:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124295/ [7124296] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124296/ [7124297] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124297/ [7124298] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124298/ [7124299] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:26:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124299/ [7124300] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 01:11:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124300/ [7124301] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124301/ [7124302] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124302/ [7124303] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 00:59:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124303/ [7124304] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124304/ [7124306] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:33:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124306/ [7124307] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:09:38 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124307/ [7124309] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124309/ [7124310] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:32:45 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124310/ [7124312] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:41:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124312/ [7124313] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124313/ [7124314] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 00:48:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124314/ [7124315] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 00:12:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124315/ [7124316] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:19:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124316/ [7124317] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:13:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124317/ [7124318] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 01:06:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124318/ [7124319] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124319/ [7124320] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124320/ [7124321] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124321/ [7124322] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124322/ [7124324] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124324/ [7124325] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:15:49 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124325/ [7124326] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:36:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124326/ [7124327] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:19:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124327/ [7124328] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124328/ [7124329] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124329/ [7124330] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:59 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124330/ [7124331] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/prometheus} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124331/ [7124332] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:50:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124332/ [7124333] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:58:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124333/ [7124334] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124334/ [7124335] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:19 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124335/ [7124337] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:39:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124337/ [7124338] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:31:13 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124338/ [7124339] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124339/ [7124340] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:08:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124340/ [7124341] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:12:26 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124341/ [7124342] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:08:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124342/ [7124344] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124344/ [7124345] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:08:00 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124345/ [7124346] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:15:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124346/ [7124347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:22:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124347/ [7124348] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:44:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124348/ [7124349] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:09:22 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124349/ [7124350] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124350/ [7124351] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:15:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124351/ [7124352] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124352/ [7124353] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:53:05 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124353/ [7124354] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:15:25 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124354/ [7124355] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:22:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124355/ [7124356] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:50 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124356/ [7124357] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 00:27:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124357/ [7124358] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:12:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124358/ [7124359] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124359/ [7124360] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124360/ [7124361] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:21:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124361/ [7124362] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124362/ [7124363] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124363/ [7124364] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:07:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124364/ [7124365] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 01:06:37 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124365/ [7124366] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124366/ [7124367] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124367/ [7124368] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124368/ [7124369] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:30:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124369/ [7124370] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:54:18 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124370/ [7124371] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124371/ [7124372] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:36:56 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124372/ [7124373] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/workunits} time: 00:11:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124373/ [7124374] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} time: 00:31:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124374/ [7124375] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124375/ [7124376] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:37:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124376/ [7124377] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} time: 03:15:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124377/ [7124378] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:23:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124378/ [7124379] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 00:16:48 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124379/ [7124381] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124381/ [7124382] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:48:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124382/ [7124383] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124383/ [7124384] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124384/ [7124385] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124385/ [7124386] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:25:12 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124386/ [7124387] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124387/ [7124388] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:24:41 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124388/ [7124389] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} time: 00:35:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124389/ [7124390] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:34:11 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124390/ [7124391] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:17:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124391/ [7124392] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:34:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124392/ [7124393] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124393/ [7124394] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:14:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124394/ [7124395] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:13:36 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124395/ [7124396] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124396/ [7124398] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124398/ [7124399] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:53 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124399/ [7124400] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:30:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124400/ [7124401] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:17:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124401/ [7124402] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124402/ [7124403] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124403/ [7124404] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:08:47 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124404/ [7124405] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:07:39 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124405/ [7124406] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:53:44 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124406/ [7124407] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:08 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124407/ [7124408] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124408/ [7124409] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} time: 00:46:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124409/ [7124410] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124410/ [7124411] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:10 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124411/ [7124412] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124412/ [7124413] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:18:51 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124413/ [7124414] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:10:27 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124414/ [7124415] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124415/ [7124416] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:33:03 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124416/ [7124417] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124417/ [7124418] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:41:31 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124418/ [7124419] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:18:26 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124419/ [7124420] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:50:35 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124420/ [7124421] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:16 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124421/ [7124422] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124422/ [7124423] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124423/ [7124424] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:25:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124424/ [7124426] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:15 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124426/ [7124427] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:43:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124427/ [7124428] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:22:32 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124428/ [7124429] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:04 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124429/ [7124430] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124430/ [7124431] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:34:19 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124431/ [7124432] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:22:30 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124432/ [7124433] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124433/ [7124434] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:45:14 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124434/ [7124435] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:33:33 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124435/ [7124436] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:26:57 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124436/ [7124437] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:34 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124437/ [7124438] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:42 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124438/ [7124439] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:40 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124439/ [7124440] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:33:02 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124440/ [7124442] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:34:58 info: http://pulpito.ceph.com/yuriw-2022-12-21_19:56:26-rados-pacific_16.2.11_RC3-distro-default-smithi/7124442/