2021-02-06T09:19:19.258 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T09:19:19.260 INFO:teuthology.results:subset: '979/9999' 2021-02-06T09:19:19.260 INFO:teuthology.results:seed: '2477' 2021-02-06T10:06:11.539 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T10:06:13.206 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-06T12:00:15.519 INFO:teuthology.results:Tests finished! gathering results... 2021-02-06T12:00:26.554 INFO:scrape:Found 60 jobs 2021-02-06T12:00:28.210 INFO:scrape:Found 5 distinct failure reasons 2021-02-06T12:00:28.211 INFO:scrape:Failure: Command failed on smithi123 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60958ea4-6863-11eb-8fda-001a4aab830c -e sha1=f02d8a6da6cd8a3848a3731a31f1b8072495d545 -- bash -c 'ceph orch ps'" 2021-02-06T12:00:28.211 INFO:scrape:2 jobs: ['5861343', '5861319'] 2021-02-06T12:00:28.212 INFO:scrape:suites intersection: ['1-tasks', 'rados/upgrade/pacific-x/parallel/{0-start', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2021-02-06T12:00:28.212 INFO:scrape:suites union: ['1-tasks', 'distro1$/{centos_8.0}', 'distro1$/{rhel_latest}', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/pacific-x/parallel/{0-start', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2021-02-06T12:00:28.213 INFO:scrape: 2021-02-06T12:00:28.214 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2021-02-06T12:00:28.214 INFO:scrape:3 jobs: ['5861323', '5861334', '5861317'] 2021-02-06T12:00:28.215 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'openstack}', 'rados', 'supported-random-distro$/{ubuntu_latest}'] 2021-02-06T12:00:28.215 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_api_tests}', 'thrashers/careful', 'thrashers/pggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2021-02-06T12:00:28.216 INFO:scrape: 2021-02-06T12:00:28.216 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-02-06T12:00:28.217 INFO:scrape:2 jobs: ['5861321', '5861344'] 2021-02-06T12:00:28.217 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-02-06T12:00:28.218 INFO:scrape:suites union: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2021-02-06T12:00:28.218 INFO:scrape: 2021-02-06T12:00:28.219 INFO:scrape:Failure: Command failed on smithi183 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' 2021-02-06T12:00:28.219 INFO:scrape:2 jobs: ['5861322', '5861345'] 2021-02-06T12:00:28.219 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/balancer', 'supported-random-distro$/{ubuntu_latest}}'] 2021-02-06T12:00:28.220 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/balancer', 'supported-random-distro$/{ubuntu_latest}}'] 2021-02-06T12:00:28.220 INFO:scrape: 2021-02-06T12:00:28.221 INFO:scrape:Failure: Command failed (workunit test mgr/balancer.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mgr/balancer.sh' 2021-02-06T12:00:28.221 INFO:scrape:2 jobs: ['5861353', '5861327'] 2021-02-06T12:00:28.222 INFO:scrape:suites intersection: ['workloads/mgr}'] 2021-02-06T12:00:28.222 INFO:scrape:suites union: ['rados/standalone/{mon_election/classic', 'rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/mgr}'] 2021-02-06T12:00:28.223 INFO:scrape: 2021-02-06T12:00:28.226 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/ failed: 11 dead: 0 running: 0 waiting: 0 queued: 0 passed: 49 Fail ================================================================= [5861317] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:21:52 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861317/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861317/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8622d193c0e5482b8a6ec0b7ca9a2e96 Command failed (workunit test rados/test.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5861319] rados/upgrade/pacific-x/parallel/{0-start 1-tasks distro1$/{rhel_latest} mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 01:59:01 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861319/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861319/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e1fd515a4ff4452aa3e0969179f23e9 Command failed on smithi090 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest- pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f791b178-6860-11eb-8fda-001a4aab830c -e sha1=f02d8a6da6cd8a3848a3731a31f1b8072495d545 -- bash -c 'ceph orch ps'" [5861321] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:18:36 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861321/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861321/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e75afc32abe84ae6a71dd4c073fe148b Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5861322] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:07:45 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861322/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861322/ sentry: https://sentry.ceph.com/organizations/ceph/?query=864f2a004f2649159f2a295ab4844460 Command failed on smithi183 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [5861323] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:25:06 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861323/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861323/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1da62ba581714b0e8e432d027d63d9ce Command failed (workunit test rados/test.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5861327] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mgr} ----------------------------------------------------------------- time: 00:06:25 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861327/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861327/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5d1646d8da4e44e480478ca50c03e349 Command failed (workunit test mgr/balancer.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mgr/balancer.sh' [5861334] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:24:37 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861334/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861334/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f70582d0eff3454487c230b9374d6a1e Command failed (workunit test rados/test.sh) on smithi074 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5861343] rados/upgrade/pacific-x/parallel/{0-start 1-tasks distro1$/{centos_8.0} mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 01:58:30 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861343/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861343/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d4fbb9648ec3436c8b7bfb79107d1fdd Command failed on smithi123 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest- pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60958ea4-6863-11eb-8fda-001a4aab830c -e sha1=f02d8a6da6cd8a3848a3731a31f1b8072495d545 -- bash -c 'ceph orch ps'" [5861344] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:18:18 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861344/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861344/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5873b8609e784e2ba092350de4ce3ee4 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5861345] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:07:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861345/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861345/ sentry: https://sentry.ceph.com/organizations/ceph/?query=93b9a306972145ff93688e3b5963bd0c Command failed on smithi197 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [5861353] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/mgr} ----------------------------------------------------------------- time: 00:09:06 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861353/ log: http://qa-proxy.ceph.com/teuthology/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861353/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fdbf150c0b5c4b688da7f7bb6879fe51 Command failed (workunit test mgr/balancer.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f02d8a6da6cd8a3848a3731a31f1b8072495d545 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mgr/balancer.sh' Pass ================================================================= [5861308] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rados_api_tests} time: 00:30:54 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861308/ [5861309] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} time: 00:15:49 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861309/ [5861310] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:29:55 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861310/ [5861311] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:10 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861311/ [5861312] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/lockdep} time: 00:25:39 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861312/ [5861313] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:20:23 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861313/ [5861314] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/scrub} time: 01:44:51 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861314/ [5861315] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:33:53 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861315/ [5861316] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:27:24 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861316/ [5861318] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:30:19 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861318/ [5861320] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:42 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861320/ [5861324] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:34 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861324/ [5861325] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:08:16 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861325/ [5861326] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:35:04 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861326/ [5861328] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:34:53 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861328/ [5861329] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:19:29 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861329/ [5861330] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:10:58 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861330/ [5861331] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:32:29 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861331/ [5861332] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_api_tests} time: 00:25:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861332/ [5861333] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:34:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861333/ [5861335] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:50 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861335/ [5861336] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/lockdep} time: 00:29:42 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861336/ [5861337] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:56 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861337/ [5861338] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} time: 00:15:01 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861338/ [5861339] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} time: 00:18:16 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861339/ [5861340] rados/cephadm/with-work/{distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:33:50 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861340/ [5861341] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:22:14 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861341/ [5861342] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:33:02 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861342/ [5861346] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:22:12 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861346/ [5861347] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_18.04 workloads/fio_4K_rand_read} time: 00:10:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861347/ [5861348] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:58 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861348/ [5861349] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:21:08 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861349/ [5861350] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861350/ [5861351] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/valgrind} time: 00:50:22 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861351/ [5861352] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:34:30 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861352/ [5861354] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:15:11 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861354/ [5861355] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:29:34 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861355/ [5861356] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:23:47 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861356/ [5861357] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:24 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861357/ [5861358] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:29:10 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861358/ [5861359] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:54 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861359/ [5861360] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:19:05 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861360/ [5861361] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:30:58 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861361/ [5861362] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:16:56 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861362/ [5861363] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:33:53 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861363/ [5861364] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:22:00 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861364/ [5861365] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:07 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861365/ [5861366] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:09 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861366/ [5861367] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:31:57 info: http://pulpito.ceph.com/kchai-2021-02-06_09:15:39-rados-wip-kefu-testing-2021-02-05-1810-distro-basic-smithi/5861367/