2020-05-05T13:46:19.586 INFO:root:teuthology version: 1.0.0-58684fc5 2020-05-05T13:46:19.587 INFO:teuthology.results:subset: '814/9999' 2020-05-05T13:46:19.588 INFO:teuthology.results:seed: '2477' 2020-05-06T00:35:26.663 INFO:root:teuthology version: 1.0.0-58684fc5 2020-05-06T00:35:26.682 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-06T12:29:50.395 INFO:teuthology.results:Tests finished! gathering results... 2020-05-06T12:29:50.892 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/ failed: 132 dead: 3 running: 0 waiting: 0 queued: 0 passed: 176 Fail ================================================================= [5024412] rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/many.yaml workloads/rados_5925.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024412/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024412/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=b9e1ebed4fe34f58b87ac7cd8d30d1e4 Command failed on smithi176 with status 1: 'sudo yum install -y kernel' [5024413] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} ----------------------------------------------------------------- time: 00:51:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024413/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024413/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=be5a6da84eae428cb2e204596e7a96c7 Test failure: setUpClass (tasks.mgr.dashboard.test_host.HostControllerTest) [5024414] rados/objectstore/{backends/filestore-idempotent.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:55:13 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024414/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024414/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0921fbb9ed554378b6ae968a3714f2f2 psutil.NoSuchProcess process no longer exists (pid=11966) [5024416] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-hybrid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 11:50:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024416/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024416/ psutil.NoSuchProcess process no longer exists (pid=7118) [5024418] rados/perf/{ceph.yaml objectstore/bluestore-basic-min-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4M_seq_read.yaml} ----------------------------------------------------------------- time: 00:10:16 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024418/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024418/ Command failed on smithi003 with status 2: 'sudo tar cz -f /tmp/tmp.4De4GDYDl7 -C /var/lib/ceph/mon/ceph-a -- .' [5024419] rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024419/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024419/ Command failed on smithi120 with status 2: 'sudo tar cz -f /tmp/tmp.dx3vNXjH92 -C /var/lib/ceph/mon/ceph-a -- .' [5024421] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 00:35:56 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024421/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024421/ SSH connection to smithi051 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024422] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects-localized.yaml} ----------------------------------------------------------------- time: 11:53:57 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024422/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024422/ SSH connection to smithi023 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --localize-reads --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [5024425] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-small-objects-many-deletes.yaml} ----------------------------------------------------------------- time: 00:36:02 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024425/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024425/ SSH connection to smithi122 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0' [5024428] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/small-objects.yaml} ----------------------------------------------------------------- time: 11:53:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024428/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024428/ psutil.NoSuchProcess process no longer exists (pid=1690) [5024429] rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4M_write.yaml} ----------------------------------------------------------------- time: 00:09:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024429/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024429/ Command failed on smithi164 with status 2: 'sudo tar cz -f /tmp/tmp.T1BfE8Lzw5 -C /var/lib/ceph/mon/ceph-a -- .' [5024432] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 11:53:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024432/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024432/ psutil.NoSuchProcess process no longer exists (pid=3992) [5024434] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_workunit_loadgen_mix.yaml} ----------------------------------------------------------------- time: 11:52:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024434/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024434/ Command failed on smithi049 with status 2: 'sudo tar cz -f /tmp/tmp.BG2TXTQ7xJ -C /var/lib/ceph/mon/ceph-a -- .' [5024437] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/mimic.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} ----------------------------------------------------------------- time: 00:41:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024437/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024437/ Command failed on smithi165 with status 125: 'sudo docker kill -s 1 ceph- bef123ba-8edd-11ea-a068-001a4aab830c-osd.5' [5024443] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_8.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} ----------------------------------------------------------------- time: 11:54:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024443/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024443/ psutil.NoSuchProcess process no longer exists (pid=15894) [5024446] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-hybrid.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} ----------------------------------------------------------------- time: 00:51:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024446/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024446/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=76c4a4a1d1364efe954e849f5cbb48f4 Test failure: test_host_create (tasks.mgr.dashboard.test_host.HostControllerNoOrchestratorTest) [5024448] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} ----------------------------------------------------------------- time: 11:54:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024448/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024448/ psutil.NoSuchProcess process no longer exists (pid=18970) [5024449] rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_omap_write.yaml} ----------------------------------------------------------------- time: 00:35:46 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024449/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024449/ unexpected end of data [5024450] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 11:54:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024450/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024450/ psutil.NoSuchProcess process no longer exists (pid=19599) [5024451] rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:50:46 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024451/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024451/ Command failed on smithi183 with status 2: 'sudo tar cz -f /tmp/tmp.KIHEaY4l74 -C /var/lib/ceph/mon/ceph-a -- .' [5024453] rados/standalone/{supported-random-distro$/{ubuntu_latest.yaml} workloads/mon.yaml} ----------------------------------------------------------------- time: 11:53:58 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024453/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024453/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=daf791afa6e648a89bc68dbe9efcd4be psutil.NoSuchProcess process no longer exists (pid=5604) [5024454] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/snaps-few-objects-localized.yaml} ----------------------------------------------------------------- time: 00:36:05 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024454/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024454/ SSH connection to smithi151 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --localize-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [5024455] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024455/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024455/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=10d4892dd67f44f5b3e6d9fa3ebf35e5 Command failed on smithi039 with status 1: 'sudo yum install -y kernel' [5024457] rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:52:53 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024457/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024457/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1a9f4c6cf26a49848dfdfec7c41811bc psutil.NoSuchProcess process no longer exists (pid=13190) [5024458] rados/singleton-nomsgr/{all/librados_hello_world.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:08:27 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024458/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024458/ Command failed on smithi016 with status 2: 'sudo tar cz -f /tmp/tmp.dOdjivQpDI -C /var/lib/ceph/mon/ceph-a -- .' [5024461] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_workunit_loadgen_mostlyread.yaml} ----------------------------------------------------------------- time: 11:52:51 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024461/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024461/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=490d0f84fe4c484da94ba949bd46f5e5 psutil.NoSuchProcess process no longer exists (pid=25593) [5024462] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 11:54:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024462/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024462/ psutil.NoSuchProcess process no longer exists (pid=19923) [5024465] rados/perf/{ceph.yaml objectstore/bluestore-low-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/sample_fio.yaml} ----------------------------------------------------------------- time: 00:12:38 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024465/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024465/ Command failed on smithi065 with status 2: 'sudo tar cz -f /tmp/tmp.B7ysUJcP8X -C /var/lib/ceph/mon/ceph-a -- .' [5024467] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/insights.yaml} ----------------------------------------------------------------- time: 00:11:33 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024467/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024467/ Command failed on smithi150 with status 2: 'sudo tar cz -f /tmp/tmp.DftVbsoT03 -C /var/lib/ceph/mon/ceph-a -- .' [5024472] rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/sync-many.yaml workloads/rados_mon_osdmap_prune.yaml} ----------------------------------------------------------------- time: 11:52:59 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024472/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024472/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1eabee044c3441bd839070432dc09283 psutil.NoSuchProcess process no longer exists (pid=18450) [5024474] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 11:50:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024474/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024474/ psutil.NoSuchProcess process no longer exists (pid=32481) [5024479] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 00:36:27 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024479/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024479/ SSH connection to smithi005 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024480] rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/sample_radosbench.yaml} ----------------------------------------------------------------- time: 00:09:46 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024480/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024480/ Command failed on smithi179 with status 2: 'sudo tar cz -f /tmp/tmp.9360oby6rb -C /var/lib/ceph/mon/ceph-a -- .' [5024485] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} ----------------------------------------------------------------- time: 00:36:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024485/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024485/ SSH connection to smithi100 was lost: "/bin/sh -c 'adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 -p unique_pool_0 bench 150 write'" [5024487] rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:08:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024487/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024487/ Command failed on smithi141 with status 2: 'sudo tar cz -f /tmp/tmp.v7LdBiN1Tw -C /var/lib/ceph/mon/ceph-a -- .' [5024488] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/readwrite.yaml} ----------------------------------------------------------------- time: 00:11:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024488/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024488/ Command failed on smithi156 with status 2: 'sudo tar cz -f /tmp/tmp.LbMTOz55o9 -C /var/lib/ceph/mon/ceph-a -- .' [5024490] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} ----------------------------------------------------------------- time: 11:45:39 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024490/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024490/ psutil.NoSuchProcess process no longer exists (pid=6236) [5024491] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 00:12:40 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024491/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024491/ Command failed on smithi016 with status 2: 'sudo tar cz -f /tmp/tmp.5NvWUC9Tnr -C /var/lib/ceph/mon/ceph-a -- .' [5024495] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/module_selftest.yaml} ----------------------------------------------------------------- time: 00:10:03 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024495/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024495/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ce855484af274ddea09c1a876c894ed8 Test failure: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) [5024497] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} ----------------------------------------------------------------- time: 11:48:45 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024497/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024497/ psutil.NoSuchProcess process no longer exists (pid=3216) [5024498] rados/standalone/{supported-random-distro$/{rhel_8.yaml} workloads/osd.yaml} ----------------------------------------------------------------- time: 04:26:15 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024498/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024498/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c95fa1c8fe8f4f1fa6d5533aef85f0ec Command failed (workunit test osd/osd-bench.sh) on smithi097 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh' [5024499] rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/cosbench_64K_read_write.yaml} ----------------------------------------------------------------- time: 11:53:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024499/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024499/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=d046a2d0bb3f4ddda6499e0d213ad3be psutil.NoSuchProcess process no longer exists (pid=31531) [5024501] rados/cephadm/workunits/{distro/ubuntu_latest.yaml task/test_cephadm.yaml} ----------------------------------------------------------------- time: 11:54:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024501/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024501/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ffd9c751449b44638d15a4933fb0335c psutil.NoSuchProcess process no longer exists (pid=431) [5024502] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/nautilus-v2only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/few.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} ----------------------------------------------------------------- time: 00:24:30 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024502/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024502/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=03212f990dc74812b3c2ad72e7c281ad reached maximum tries (180) after waiting for 180 seconds [5024505] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} ----------------------------------------------------------------- time: 01:12:42 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024505/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024505/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=22653ce343df4d6fbfe73cbc619a98e7 Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh' [5024509] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 00:36:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024509/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024509/ SSH connection to smithi058 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024514] rados/perf/{ceph.yaml objectstore/bluestore-basic-min-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/cosbench_64K_write.yaml} ----------------------------------------------------------------- time: 11:50:57 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024514/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024514/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4ec6818256ff4b2b979acaf4469c5e17 psutil.NoSuchProcess process no longer exists (pid=25180) [5024520] rados/objectstore/{backends/objectcacher-stress.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:53:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024520/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024520/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=138a7ad00324484b8a77362d44ae31c1 psutil.NoSuchProcess process no longer exists (pid=23925) [5024522] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} ----------------------------------------------------------------- time: 00:36:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024522/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024522/ SSH connection to smithi039 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op cache_flush 50 --op cache_try_flush 50 --op cache_evict 50 --op write_excl 50 --pool base' [5024524] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 00:11:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024524/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024524/ Command failed on smithi052 with status 2: 'sudo tar cz -f /tmp/tmp.wtM5BoEiRa -C /var/lib/ceph/mon/ceph-a -- .' [5024527] rados/cephadm/with-work/{distro/ubuntu_latest.yaml fixed-2.yaml mode/root.yaml msgr/async-v1only.yaml start.yaml tasks/rados_api_tests.yaml} ----------------------------------------------------------------- time: 11:52:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024527/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024527/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3b2fa3f354c048f096f4a546c4dac8d9 psutil.NoSuchProcess process no longer exists (pid=1940) [5024530] rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:54:15 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024530/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024530/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=6a3bbe9616b04d258cc9cd3efb496f8c psutil.NoSuchProcess process no longer exists (pid=7646) [5024532] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} ----------------------------------------------------------------- time: 11:52:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024532/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024532/ SSH connection to smithi114 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op cache_flush 50 --op cache_try_flush 50 --op cache_evict 50 --op write_excl 50 --pool base' [5024534] rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:07:16 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024534/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024534/ Command failed on smithi067 with status 2: 'sudo tar cz -f /tmp/tmp.vEnJXh41o3 -C /var/lib/ceph/mon/ceph-a -- .' [5024536] rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/fio_4K_rand_read.yaml} ----------------------------------------------------------------- time: 00:12:13 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024536/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024536/ Command failed on smithi118 with status 2: 'sudo tar cz -f /tmp/tmp.XbPNmgmSKm -C /var/lib/ceph/mon/ceph-a -- .' [5024537] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/minsize_recovery.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} ----------------------------------------------------------------- time: 11:54:28 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024537/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024537/ psutil.NoSuchProcess process no longer exists (pid=21547) [5024538] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/on.yaml distro$/{centos_7.6.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} ----------------------------------------------------------------- time: 00:24:32 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024538/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024538/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=506b54a4b9794a01a789dfb11c3c3dc3 reached maximum tries (180) after waiting for 180 seconds [5024539] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 00:36:23 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024539/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024539/ SSH connection to smithi037 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024542] rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/force-sync-many.yaml workloads/snaps-few-objects.yaml} ----------------------------------------------------------------- time: 00:35:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024542/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024542/ SSH connection to smithi136 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max- ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [5024543] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/scrub_test.yaml} ----------------------------------------------------------------- time: 00:09:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024543/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024543/ Command failed on smithi071 with status 2: 'sudo tar cz -f /tmp/tmp.s9J7LesWMW -C /var/lib/ceph/mon/ceph-a -- .' [5024550] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} ----------------------------------------------------------------- time: 11:53:51 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024550/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024550/ SSH connection to smithi173 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024552] rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/fio_4K_rand_rw.yaml} ----------------------------------------------------------------- time: 00:13:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024552/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024552/ Command failed on smithi130 with status 2: 'sudo tar cz -f /tmp/tmp.aTRcv598cm -C /var/lib/ceph/mon/ceph-a -- .' [5024553] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} ----------------------------------------------------------------- time: 11:53:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024553/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024553/ psutil.NoSuchProcess process no longer exists (pid=28880) [5024554] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-hybrid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 11:35:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024554/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024554/ psutil.NoSuchProcess process no longer exists (pid=20831) [5024555] rados/singleton-nomsgr/{all/recovery-unfound-found.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:36:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024555/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024555/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a06a88d61fe64d86b49e05cbc8200fab [Errno None] Unable to connect to port 22 on 172.21.15.160 [5024558] rados/objectstore/{backends/objectstore.yaml supported-random-distro$/{centos_8.yaml}} ----------------------------------------------------------------- time: 02:06:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024558/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024558/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=cabff41391774fca9ddb313cb67541c4 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-filestore 20 --debug- bluestore 20" ceph_test_objectstore --gtest_filter=-*/3 --gtest_catch_exceptions=0\'' [5024561] rados/singleton/{all/max-pg-per-osd.from-primary.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:11:03 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024561/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024561/ Command failed on smithi151 with status 2: 'sudo tar cz -f /tmp/tmp.vqTaI2KfhY -C /var/lib/ceph/mon/ceph-a -- .' [5024565] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} tasks/rados_api_tests.yaml} ----------------------------------------------------------------- time: 03:31:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024565/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024565/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c60426f6bca04fbf9801307b26a40cb2 Command failed (workunit test rados/test.sh) on smithi019 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5024567] rados/perf/{ceph.yaml objectstore/bluestore-low-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/fio_4M_rand_read.yaml} ----------------------------------------------------------------- time: 00:10:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024567/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024567/ Command failed on smithi148 with status 2: 'sudo tar cz -f /tmp/tmp.aKWGADgJnK -C /var/lib/ceph/mon/ceph-a -- .' [5024569] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} ----------------------------------------------------------------- time: 11:53:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024569/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024569/ SSH connection to smithi118 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024575] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/dedup_tier.yaml} ----------------------------------------------------------------- time: 11:52:52 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024575/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024575/ psutil.NoSuchProcess process no longer exists (pid=9428) [5024576] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 11:51:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024576/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024576/ psutil.NoSuchProcess process no longer exists (pid=16567) [5024579] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} ----------------------------------------------------------------- time: 00:36:38 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024579/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024579/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3f7a95b1bc1b40a5ad9767c75625b00c SSH connection to smithi081 was lost: "/bin/sh -c 'adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 -p unique_pool_0 bench 150 write'" [5024580] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} ----------------------------------------------------------------- time: 00:36:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024580/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024580/ SSH connection to smithi041 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 50 --op append_excl 50 --pool unique_pool_0' [5024581] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 05:48:23 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024581/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024581/ Command failed on smithi119 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json' [5024583] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} ----------------------------------------------------------------- time: 11:56:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024583/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024583/ psutil.NoSuchProcess process no longer exists (pid=24451) [5024584] rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/fio_4M_rand_rw.yaml} ----------------------------------------------------------------- time: 00:10:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024584/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024584/ Command failed on smithi130 with status 2: 'sudo tar cz -f /tmp/tmp.UbFQmsJotG -C /var/lib/ceph/mon/ceph-a -- .' [5024585] rados/singleton/{all/mon-config-key-caps.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:08:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024585/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024585/ Command failed on smithi068 with status 2: 'sudo tar cz -f /tmp/tmp.UW92BLzrhI -C /var/lib/ceph/mon/ceph-a -- .' [5024587] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/workunits.yaml} ----------------------------------------------------------------- time: 00:08:06 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024587/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024587/ Command failed on smithi058 with status 2: 'sudo tar cz -f /tmp/tmp.hElpAzhO6B -C /var/lib/ceph/mon/ceph-a -- .' [5024593] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/nautilus.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-octopus.yaml 8-workload/{rbd-python.yaml snaps-many-objects.yaml} bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:03:14 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024593/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024593/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e035483a879b4d7bab256937cdae22dc Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref= nautilus [5024599] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} ----------------------------------------------------------------- time: 11:51:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024599/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024599/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a0698135f84d405bb00897448688b0fb Command failed (workunit test rados/test.sh) on smithi047 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5024601] rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:45:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024601/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024601/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=82c4f160ed2e435a959e07eae6bf3eb3 psutil.NoSuchProcess process no longer exists (pid=26938) [5024602] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 11:46:03 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024602/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024602/ psutil.NoSuchProcess process no longer exists (pid=27160) [5024603] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} ----------------------------------------------------------------- time: 11:52:58 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024603/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024603/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8ea2ac8382a646d4bf3ad2d79da16109 Command failed (workunit test rados/test.sh) on smithi164 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5024605] rados/perf/{ceph.yaml objectstore/bluestore-basic-min-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/fio_4M_rand_write.yaml} ----------------------------------------------------------------- time: 00:11:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024605/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024605/ Command failed on smithi033 with status 2: 'sudo tar cz -f /tmp/tmp.TCQu4UmHYO -C /var/lib/ceph/mon/ceph-a -- .' [5024606] rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/one.yaml workloads/rados_5925.yaml} ----------------------------------------------------------------- time: 00:08:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024606/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024606/ Command failed on smithi100 with status 2: 'sudo tar cz -f /tmp/tmp.gVFqiDVsI3 -C /var/lib/ceph/mon/ceph-a -- .' [5024608] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/on.yaml distro$/{centos_7.6.yaml} msgr-failures/few.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} ----------------------------------------------------------------- time: 00:37:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024608/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024608/ Command failed on smithi148 with status 125: 'sudo docker kill -s 1 ceph-f6a62aba-8ee9-11ea-a068-001a4aab830c-osd.9' [5024611] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 00:36:02 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024611/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024611/ SSH connection to smithi141 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0' [5024615] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-small-objects-balanced.yaml} ----------------------------------------------------------------- time: 11:53:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024615/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024615/ psutil.NoSuchProcess process no longer exists (pid=4135) [5024618] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} ----------------------------------------------------------------- time: 11:56:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024618/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024618/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f2d49f23329548c7b7f0b67a00ebeb05 reached maximum tries (500) after waiting for 3000 seconds [5024621] rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4K_rand_read.yaml} ----------------------------------------------------------------- time: 00:11:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024621/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024621/ Command failed on smithi095 with status 2: 'sudo tar cz -f /tmp/tmp.XHASXkW5dR -C /var/lib/ceph/mon/ceph-a -- .' [5024624] rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:51:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024624/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024624/ Command failed on smithi153 with status 2: 'sudo tar cz -f /tmp/tmp.b6kxVqhzE8 -C /var/lib/ceph/mon/ceph-a -- .' [5024627] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} ----------------------------------------------------------------- time: 11:54:40 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024627/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024627/ SSH connection to smithi125 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 50 --op append_excl 50 --pool unique_pool_0' [5024629] rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:12:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024629/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024629/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=76d4907cea7e4f73a58678850f88d25e Command failed on smithi112 with status 4: "sudo bash -c 'nmcli connection up ceph-brx'" [5024634] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} ----------------------------------------------------------------- time: 04:49:45 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024634/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024634/ saw valgrind issues [5024635] rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/sync-many.yaml workloads/rados_api_tests.yaml} ----------------------------------------------------------------- time: 00:52:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024635/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024635/ "2020-05-05T23:21:31.515828+0000 mon.f (mon.0) 93 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [5024638] rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4K_seq_read.yaml} ----------------------------------------------------------------- time: 00:16:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024638/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024638/ Command failed on smithi168 with status 2: 'sudo tar cz -f /tmp/tmp.sukLWML4ge -C /var/lib/ceph/mon/ceph-a -- .' [5024641] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} ----------------------------------------------------------------- time: 11:52:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024641/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024641/ Command failed on smithi058 with status 2: 'sudo tar cz -f /tmp/tmp.JZR5TjUIWD -C /var/lib/ceph/mon/ceph-a -- .' [5024642] rados/singleton/{all/peer.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:07:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024642/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024642/ Command failed on smithi037 with status 2: 'sudo tar cz -f /tmp/tmp.ed6Jl96t4N -C /var/lib/ceph/mon/ceph-a -- .' [5024643] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 00:17:34 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024643/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024643/ Command failed on smithi099 with status 2: 'sudo tar cz -f /tmp/tmp.gbdxN1KHUI -C /var/lib/ceph/mon/ceph-a -- .' [5024644] rados/cephadm/smoke-roleless/{distro/centos_latest.yaml start.yaml} ----------------------------------------------------------------- time: 00:03:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024644/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024644/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ee606ede3cef454e80ef1df01e2cff67 empty file [5024645] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} ----------------------------------------------------------------- time: 11:54:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024645/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024645/ psutil.NoSuchProcess process no longer exists (pid=26946) [5024651] rados/cephadm/workunits/{distro/ubuntu_latest.yaml task/test_cephadm_repos.yaml} ----------------------------------------------------------------- time: 00:03:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024651/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024651/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=81f87ec4b3b84f0885ad0d99509c6894 Command failed (workunit test cephadm/test_repos.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh' [5024652] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} ----------------------------------------------------------------- time: 00:17:54 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024652/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024652/ Command failed on smithi081 with status 2: 'sudo tar cz -f /tmp/tmp.GXjb2N3bC9 -C /var/lib/ceph/mon/ceph-a -- .' [5024653] rados/perf/{ceph.yaml objectstore/bluestore-low-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4M_rand_read.yaml} ----------------------------------------------------------------- time: 00:11:02 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024653/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024653/ Command failed on smithi082 with status 2: 'sudo tar cz -f /tmp/tmp.uudoIPGLUt -C /var/lib/ceph/mon/ceph-a -- .' [5024658] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} ----------------------------------------------------------------- time: 00:36:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024658/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024658/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1c2ae8ed3e954182ab90a64f7a41b67b Test failure: setUpClass (tasks.mgr.dashboard.test_crush_rule.CrushRuleTest) [5024660] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/set-chunk-promote-flush.yaml} ----------------------------------------------------------------- time: 11:53:56 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024660/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024660/ psutil.NoSuchProcess process no longer exists (pid=10876) [5024661] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_8.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 11:54:34 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024661/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024661/ psutil.NoSuchProcess process no longer exists (pid=11518) [5024664] rados/cephadm/with-work/{distro/ubuntu_latest.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v2only.yaml start.yaml tasks/rados_python.yaml} ----------------------------------------------------------------- time: 11:55:10 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024664/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024664/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=b638a6534ee6484280fd63a2de855f63 [Errno None] Unable to connect to port 22 on 172.21.15.119 [5024668] rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4M_seq_read.yaml} ----------------------------------------------------------------- time: 00:10:13 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024668/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024668/ Command failed on smithi131 with status 2: 'sudo tar cz -f /tmp/tmp.p3QR3OY8Vq -C /var/lib/ceph/mon/ceph-a -- .' [5024669] rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 11:53:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024669/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024669/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4b36869882ab4bd4872265b9926c4c0f psutil.NoSuchProcess process no longer exists (pid=32125) [5024670] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 11:54:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024670/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024670/ psutil.NoSuchProcess process no longer exists (pid=5689) [5024672] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} ----------------------------------------------------------------- time: 11:54:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024672/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024672/ psutil.NoSuchProcess process no longer exists (pid=9103) [5024681] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/mimic-v1only.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/on.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} ----------------------------------------------------------------- time: 01:02:41 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024681/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024681/ Command failed on smithi190 with status 125: 'sudo docker kill -s 1 ceph-a696adc2-8f2c-11ea-a068-001a4aab830c-osd.6' [5024685] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml objectstore/bluestore-hybrid.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-many-deletes.yaml} ----------------------------------------------------------------- time: 11:54:11 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024685/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024685/ psutil.NoSuchProcess process no longer exists (pid=16795) [5024695] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects-balanced.yaml} ----------------------------------------------------------------- time: 11:55:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024695/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024695/ psutil.NoSuchProcess process no longer exists (pid=19662) [5024701] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml msgr/async-v1only.yaml no_pools.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/mon_clock_no_skews.yaml} ----------------------------------------------------------------- time: 00:14:50 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024701/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024701/ Command failed on smithi129 with status 2: 'sudo tar cz -f /tmp/tmp.MmV3gaOcKl -C /var/lib/ceph/mon/ceph-a -- .' [5024703] rados/singleton/{all/rebuild-mondb.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:35:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024703/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024703/ unexpected end of data [5024704] rados/perf/{ceph.yaml objectstore/bluestore-basic-min-osd-mem-target.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_4M_write.yaml} ----------------------------------------------------------------- time: 00:09:28 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024704/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024704/ Command failed on smithi167 with status 2: 'sudo tar cz -f /tmp/tmp.MLHm1gjtY6 -C /var/lib/ceph/mon/ceph-a -- .' [5024710] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/small-objects-localized.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024710/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024710/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e4f3011ac7064e2685e33ed0588a5219 Command failed on smithi081 with status 1: 'sudo yum install -y kernel' [5024717] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml} ----------------------------------------------------------------- time: 09:40:58 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024717/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024717/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=075543364be542f4991372b93bd8f3ea Command failed (workunit test rados/test.sh) on smithi095 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b82b688c49e1ace1d53d03c387a7b08147555f10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5024727] rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml} ----------------------------------------------------------------- time: 00:51:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024727/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024727/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4c39ed4faf034377bd030e55562c1c0d [Errno None] Unable to connect to port 22 on 172.21.15.73 [5024731] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} ----------------------------------------------------------------- time: 11:49:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024731/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024731/ Command failed on smithi044 with status 2: 'sudo tar cz -f /tmp/tmp.d5qhZpI5VH -C /var/lib/ceph/mon/ceph-a -- .' [5024733] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} ----------------------------------------------------------------- time: 11:54:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024733/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024733/ psutil.NoSuchProcess process no longer exists (pid=6336) [5024741] rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/radosbench_omap_write.yaml} ----------------------------------------------------------------- time: 03:08:42 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024741/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024741/ reached maximum tries (1550) after waiting for 9300 seconds [5024743] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/small-objects.yaml} ----------------------------------------------------------------- time: 11:54:10 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024743/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024743/ psutil.NoSuchProcess process no longer exists (pid=15722) [5024745] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/mimic.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/few.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} ----------------------------------------------------------------- time: 01:28:35 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024745/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024745/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1ebca89b77604072817f6ddec4d8f29c reached maximum tries (500) after waiting for 3000 seconds [5024751] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{rhel_8.yaml} tasks/module_selftest.yaml} ----------------------------------------------------------------- time: 00:31:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024751/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024751/ "2020-05-06T00:41:21.441327+0000 mds.b (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi144:z (7329), after 304.322 seconds" in cluster log [5024757] rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} ----------------------------------------------------------------- time: 00:36:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024757/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024757/ SSH connection to smithi143 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no- omap --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 50 --op append_excl 50 --pool unique_pool_0' [5024767] rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml ubuntu_latest.yaml workloads/sample_fio.yaml} ----------------------------------------------------------------- time: 00:14:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024767/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024767/ Command failed on smithi005 with status 2: 'sudo tar cz -f /tmp/tmp.Syw32Ke96m -C /var/lib/ceph/mon/ceph-a -- .' [5024778] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects-localized.yaml} ----------------------------------------------------------------- time: 11:54:42 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024778/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024778/ psutil.NoSuchProcess process no longer exists (pid=22228) [5024780] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} ----------------------------------------------------------------- time: 11:45:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024780/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024780/ psutil.NoSuchProcess process no longer exists (pid=32312) Dead ================================================================= [5024463] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024463/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024463/ [5024475] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024475/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024475/ [5024518] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024518/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024518/ Pass ================================================================= [5024411] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/small-objects-balanced.yaml} time: 00:20:52 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024411/ [5024415] rados/cephadm/with-work/{distro/centos_8.0.yaml fixed-2.yaml mode/root.yaml msgr/async-v1only.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:29:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024415/ [5024417] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} time: 00:16:03 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024417/ [5024420] rados/cephadm/smoke/{distro/ubuntu_latest.yaml fixed-2.yaml start.yaml} time: 00:13:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024420/ [5024423] rados/singleton-nomsgr/{all/large-omap-object-warnings.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:08:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024423/ [5024424] rados/cephadm/smoke-roleless/{distro/ubuntu_latest.yaml start.yaml} time: 00:12:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024424/ [5024426] rados/singleton/{all/thrash-backfill-full.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 01:26:18 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024426/ [5024427] rados/cephadm/workunits/{distro/ubuntu_18.04_podman.yaml task/test_cephadm_repos.yaml} time: 00:03:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024427/ [5024430] rados/cephadm/with-work/{distro/centos_latest.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v2only.yaml start.yaml tasks/rados_python.yaml} time: 00:19:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024430/ [5024431] rados/singleton/{all/thrash-eio.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:29:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024431/ [5024433] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml msgr/async-v2only.yaml no_pools.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_clock_with_skews.yaml} time: 00:06:59 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024433/ [5024435] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{centos_8.yaml} tasks/failover.yaml} time: 00:11:21 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024435/ [5024436] rados/cephadm/orchestrator_cli/{2-node-mgr.yaml orchestrator_cli.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:13:54 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024436/ [5024438] rados/singleton-nomsgr/{all/lazy_omap_stats_output.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:08:18 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024438/ [5024439] rados/cephadm/smoke/{distro/centos_7.yaml fixed-2.yaml start.yaml} time: 00:13:06 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024439/ [5024440] rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:21:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024440/ [5024441] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-async-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects-balanced.yaml} time: 00:25:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024441/ [5024442] rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/one.yaml workloads/rados_api_tests.yaml} time: 00:28:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024442/ [5024444] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} time: 00:26:21 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024444/ [5024445] rados/cephadm/smoke-roleless/{distro/centos_7.yaml start.yaml} time: 00:13:03 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024445/ [5024447] rados/objectstore/{backends/fusestore.yaml supported-random-distro$/{ubuntu_latest.yaml}} time: 00:06:39 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024447/ [5024452] rados/cephadm/upgrade/{1-start.yaml 2-start-upgrade.yaml 3-wait.yaml distro$/{ubuntu_18.04.yaml} fixed-2.yaml} time: 00:16:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024452/ [5024456] rados/cephadm/with-work/{distro/rhel_8.0.yaml fixed-2.yaml mode/root.yaml msgr/async.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:33:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024456/ [5024459] rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:09:38 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024459/ [5024460] rados/cephadm/smoke/{distro/centos_8.0.yaml fixed-2.yaml start.yaml} time: 00:10:15 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024460/ [5024464] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml msgr/async.yaml no_pools.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_recovery.yaml} time: 00:23:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024464/ [5024466] rados/cephadm/smoke-roleless/{distro/centos_8.0.yaml start.yaml} time: 00:10:05 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024466/ [5024468] rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:08:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024468/ [5024469] rados/cephadm/workunits/{distro/ubuntu_18.04_podman.yaml task/test_adoption.yaml} time: 00:09:18 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024469/ [5024470] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus-v1only.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/on.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} time: 00:21:53 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024470/ [5024471] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} time: 00:14:30 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024471/ [5024473] rados/cephadm/with-work/{distro/rhel_latest.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v1only.yaml start.yaml tasks/rados_python.yaml} time: 00:20:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024473/ [5024476] rados/singleton-nomsgr/{all/msgr.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} time: 00:17:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024476/ [5024477] rados/singleton/{all/deduptool.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:09:39 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024477/ [5024478] rados/cephadm/smoke/{distro/centos_latest.yaml fixed-2.yaml start.yaml} time: 00:09:15 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024478/ [5024481] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{rhel_8.yaml} tasks/dashboard.yaml} time: 01:05:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024481/ [5024482] rados/objectstore/{backends/keyvaluedb.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:13:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024482/ [5024483] rados/valgrind-leaks/{1-start.yaml 2-inject-leak/osd.yaml centos_latest.yaml} time: 00:16:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024483/ [5024484] rados/cephadm/smoke-roleless/{distro/centos_latest.yaml start.yaml} time: 00:09:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024484/ [5024486] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=clay-k=4-m=2.yaml} time: 00:29:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024486/ [5024489] rados/cephadm/with-work/{distro/ubuntu_18.04.yaml fixed-2.yaml mode/root.yaml msgr/async-v2only.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:30:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024489/ [5024492] rados/cephadm/smoke/{distro/rhel_7.yaml fixed-2.yaml start.yaml} time: 00:14:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024492/ [5024493] rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:21:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024493/ [5024494] rados/singleton/{all/divergent_priors2.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:10:50 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024494/ [5024496] rados/multimon/{clusters/3.yaml msgr-failures/few.yaml msgr/async-v1only.yaml no_pools.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} tasks/mon_clock_no_skews.yaml} time: 00:08:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024496/ [5024500] rados/cephadm/smoke-roleless/{distro/rhel_7.yaml start.yaml} time: 00:14:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024500/ [5024503] rados/singleton/{all/dump-stuck.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:08:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024503/ [5024504] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} time: 00:09:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024504/ [5024506] rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/sync.yaml workloads/rados_mon_workunits.yaml} time: 00:27:42 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024506/ [5024507] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} time: 00:17:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024507/ [5024508] rados/cephadm/with-work/{distro/ubuntu_18.04_podman.yaml fixed-2.yaml mode/packaged.yaml msgr/async.yaml start.yaml tasks/rados_python.yaml} time: 00:21:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024508/ [5024510] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/repair_test.yaml} time: 00:15:41 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024510/ [5024511] rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:30:53 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024511/ [5024512] rados/cephadm/smoke/{distro/rhel_8.0.yaml fixed-2.yaml start.yaml} time: 00:12:58 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024512/ [5024513] rados/singleton-nomsgr/{all/osd_stale_reads.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:19:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024513/ [5024516] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/minsize_recovery.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} time: 00:24:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024516/ [5024523] rados/cephadm/smoke-roleless/{distro/rhel_8.0.yaml start.yaml} time: 00:13:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024523/ [5024526] rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:08:46 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024526/ [5024528] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-hybrid.yaml supported-random-distro$/{rhel_8.yaml} tasks/progress.yaml} time: 00:16:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024528/ [5024529] rados/multimon/{clusters/6.yaml msgr-failures/many.yaml msgr/async-v2only.yaml no_pools.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_clock_with_skews.yaml} time: 00:06:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024529/ [5024531] rados/cephadm/smoke/{distro/rhel_latest.yaml fixed-2.yaml start.yaml} time: 00:09:54 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024531/ [5024533] rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:30:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024533/ [5024535] rados/cephadm/smoke-roleless/{distro/rhel_latest.yaml start.yaml} time: 00:10:02 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024535/ [5024540] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} time: 00:16:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024540/ [5024541] rados/cephadm/workunits/{distro/centos_latest.yaml task/test_cephadm_repos.yaml} time: 00:03:33 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024541/ [5024544] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-snaps-balanced.yaml} time: 00:17:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024544/ [5024545] rados/singleton/{all/lost-unfound.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:28:00 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024545/ [5024546] rados/cephadm/smoke/{distro/ubuntu_18.04.yaml fixed-2.yaml start.yaml} time: 00:12:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024546/ [5024547] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-hybrid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} time: 00:24:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024547/ [5024548] rados/standalone/{supported-random-distro$/{centos_8.yaml} workloads/scrub.yaml} time: 01:12:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024548/ [5024549] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04.yaml start.yaml} time: 00:11:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024549/ [5024551] rados/singleton/{all/max-pg-per-osd.from-mon.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:09:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024551/ [5024556] rados/cephadm/with-work/{distro/centos_8.0.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v2only.yaml start.yaml tasks/rados_python.yaml} time: 00:20:01 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024556/ [5024557] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{rhel_8.yaml} tasks/dashboard.yaml} time: 01:04:13 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024557/ [5024559] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{rhel_8.yaml} tasks/prometheus.yaml} time: 00:12:27 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024559/ [5024560] rados/cephadm/smoke/{distro/ubuntu_18.04_podman.yaml fixed-2.yaml start.yaml} time: 00:12:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024560/ [5024562] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml msgr/async.yaml no_pools.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_recovery.yaml} time: 00:12:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024562/ [5024563] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache.yaml} time: 00:14:30 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024563/ [5024564] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman.yaml start.yaml} time: 00:11:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024564/ [5024566] rados/cephadm/with-work/{distro/centos_latest.yaml fixed-2.yaml mode/root.yaml msgr/async.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:26:53 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024566/ [5024568] rados/singleton/{all/max-pg-per-osd.from-replica.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:14:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024568/ [5024570] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} time: 00:35:23 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024570/ [5024571] rados/singleton-nomsgr/{all/version-number-sanity.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:10:16 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024571/ [5024572] rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/many.yaml workloads/pool-create-delete.yaml} time: 00:19:27 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024572/ [5024573] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/octopus.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/none.yaml thrashosds-health.yaml workloads/rbd_cls.yaml} time: 00:19:19 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024573/ [5024574] rados/cephadm/workunits/{distro/centos_latest.yaml task/test_adoption.yaml} time: 00:07:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024574/ [5024577] rados/singleton/{all/mon-auth-caps.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:08:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024577/ [5024578] rados/cephadm/smoke/{distro/ubuntu_latest.yaml fixed-2.yaml start.yaml} time: 00:12:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024578/ [5024582] rados/cephadm/smoke-roleless/{distro/ubuntu_latest.yaml start.yaml} time: 00:13:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024582/ [5024586] rados/cephadm/with-work/{distro/rhel_8.0.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v1only.yaml start.yaml tasks/rados_python.yaml} time: 00:23:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024586/ [5024588] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/dashboard.yaml} time: 01:03:21 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024588/ [5024589] rados/objectstore/{backends/alloc-hint.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:09:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024589/ [5024590] rados/rest/{mgr-restful.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:10:30 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024590/ [5024591] rados/singleton-nomsgr/{all/admin_socket_output.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:14:20 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024591/ [5024592] rados/standalone/{supported-random-distro$/{rhel_8.yaml} workloads/crush.yaml} time: 00:12:06 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024592/ [5024594] rados/valgrind-leaks/{1-start.yaml 2-inject-leak/mon.yaml centos_latest.yaml} time: 00:19:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024594/ [5024595] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml msgr/async-v1only.yaml no_pools.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} tasks/mon_clock_no_skews.yaml} time: 00:09:16 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024595/ [5024596] rados/cephadm/orchestrator_cli/{2-node-mgr.yaml orchestrator_cli.yaml supported-random-distro$/{centos_8.yaml}} time: 00:11:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024596/ [5024597] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/rados_cls_all.yaml} time: 00:14:30 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024597/ [5024598] rados/singleton/{all/mon-config-keys.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:12:28 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024598/ [5024600] rados/cephadm/smoke/{distro/centos_7.yaml fixed-2.yaml start.yaml} time: 00:13:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024600/ [5024604] rados/cephadm/smoke-roleless/{distro/centos_7.yaml start.yaml} time: 00:13:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024604/ [5024607] rados/singleton/{all/mon-config.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:11:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024607/ [5024609] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/radosbench-high-concurrency.yaml} time: 00:56:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024609/ [5024610] rados/cephadm/upgrade/{1-start.yaml 2-start-upgrade.yaml 3-wait.yaml distro$/{centos_7.yaml} fixed-2.yaml} time: 00:18:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024610/ [5024612] rados/singleton-nomsgr/{all/balancer.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:11:29 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024612/ [5024613] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} time: 00:11:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024613/ [5024614] rados/cephadm/with-work/{distro/rhel_latest.yaml fixed-2.yaml mode/root.yaml msgr/async-v2only.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:30:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024614/ [5024616] rados/singleton/{all/osd-backfill.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:15:51 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024616/ [5024617] rados/cephadm/workunits/{distro/ubuntu_18.04_podman.yaml task/test_cephadm.yaml} time: 00:18:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024617/ [5024619] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{rhel_8.yaml} tasks/crash.yaml} time: 00:12:26 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024619/ [5024620] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} tasks/rados_python.yaml} time: 00:15:41 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024620/ [5024622] rados/cephadm/smoke/{distro/centos_8.0.yaml fixed-2.yaml start.yaml} time: 00:10:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024622/ [5024623] rados/multimon/{clusters/3.yaml msgr-failures/few.yaml msgr/async-v2only.yaml no_pools.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_clock_with_skews.yaml} time: 00:06:11 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024623/ [5024625] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{centos_8.yaml} tasks/dashboard.yaml} time: 01:01:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024625/ [5024626] rados/objectstore/{backends/ceph_objectstore_tool.yaml supported-random-distro$/{centos_8.yaml}} time: 00:12:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024626/ [5024628] rados/cephadm/smoke-roleless/{distro/centos_8.0.yaml start.yaml} time: 00:10:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024628/ [5024630] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-async-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/redirect.yaml} time: 00:18:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024630/ [5024631] rados/cephadm/with-work/{distro/ubuntu_18.04.yaml fixed-2.yaml mode/packaged.yaml msgr/async.yaml start.yaml tasks/rados_python.yaml} time: 00:27:51 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024631/ [5024632] rados/singleton/{all/osd-recovery.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:15:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024632/ [5024633] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} time: 00:12:51 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024633/ [5024636] rados/standalone/{supported-random-distro$/{rhel_8.yaml} workloads/erasure-code.yaml} time: 00:39:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024636/ [5024637] rados/cephadm/smoke/{distro/centos_latest.yaml fixed-2.yaml start.yaml} time: 00:09:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024637/ [5024639] rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} time: 00:28:21 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024639/ [5024640] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} time: 00:23:50 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024640/ [5024646] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/rados_stress_watch.yaml} time: 00:18:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024646/ [5024647] rados/cephadm/with-work/{distro/ubuntu_18.04_podman.yaml fixed-2.yaml mode/root.yaml msgr/async-v1only.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:37:33 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024647/ [5024648] rados/singleton-nomsgr/{all/ceph-kvstore-tool.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:13:54 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024648/ [5024649] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/failover.yaml} time: 00:13:57 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024649/ [5024650] rados/singleton/{all/pg-autoscaler.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:13:31 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024650/ [5024654] rados/multimon/{clusters/6.yaml msgr-failures/many.yaml msgr/async.yaml no_pools.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_recovery.yaml} time: 00:12:43 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024654/ [5024655] rados/cephadm/smoke/{distro/rhel_7.yaml fixed-2.yaml start.yaml} time: 00:14:08 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024655/ [5024656] rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-lz4.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:12:45 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024656/ [5024657] rados/cephadm/smoke-roleless/{distro/rhel_7.yaml start.yaml} time: 00:14:33 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024657/ [5024659] rados/objectstore/{backends/filejournal.yaml supported-random-distro$/{centos_8.yaml}} time: 00:08:48 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024659/ [5024662] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} time: 00:15:17 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024662/ [5024663] rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:07:44 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024663/ [5024665] rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/sync.yaml workloads/rados_mon_osdmap_prune.yaml} time: 00:43:06 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024665/ [5024666] rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:24:28 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024666/ [5024667] rados/cephadm/smoke/{distro/rhel_8.0.yaml fixed-2.yaml start.yaml} time: 00:13:22 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024667/ [5024674] rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} time: 00:17:24 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024674/ [5024676] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-zstd.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/rados_striper.yaml} time: 00:12:15 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024676/ [5024679] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} time: 00:20:02 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024679/ [5024683] rados/cephadm/smoke-roleless/{distro/rhel_8.0.yaml start.yaml} time: 00:12:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024683/ [5024687] rados/singleton/{all/random-eio.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp-zlib.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:29:09 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024687/ [5024689] rados/mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{rhel_8.yaml} tasks/insights.yaml} time: 00:17:47 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024689/ [5024691] rados/cephadm/with-work/{distro/centos_8.0.yaml fixed-2.yaml mode/root.yaml msgr/async.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:32:34 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024691/ [5024693] rados/standalone/{supported-random-distro$/{rhel_8.yaml} workloads/mgr.yaml} time: 00:16:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024693/ [5024697] rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:10:11 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024697/ [5024699] rados/cephadm/workunits/{distro/ubuntu_latest.yaml task/test_adoption.yaml} time: 00:08:14 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024699/ [5024706] rados/cephadm/smoke/{distro/rhel_latest.yaml fixed-2.yaml start.yaml} time: 00:10:32 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024706/ [5024708] rados/cephadm/smoke-roleless/{distro/rhel_latest.yaml start.yaml} time: 00:09:59 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024708/ [5024713] rados/singleton/{all/recovery-preemption.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:23:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024713/ [5024715] rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-hybrid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_8.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} time: 00:17:42 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024715/ [5024719] rados/dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{rhel_8.yaml} tasks/dashboard.yaml} time: 01:07:52 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024719/ [5024721] rados/objectstore/{backends/filestore-idempotent-aio-journal.yaml supported-random-distro$/{centos_8.yaml}} time: 02:19:55 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024721/ [5024723] rados/valgrind-leaks/{1-start.yaml 2-inject-leak/none.yaml centos_latest.yaml} time: 03:21:57 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024723/ [5024725] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/rados_workunit_loadgen_big.yaml} time: 00:31:36 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024725/ [5024729] rados/cephadm/with-work/{distro/centos_latest.yaml fixed-2.yaml mode/packaged.yaml msgr/async-v1only.yaml start.yaml tasks/rados_python.yaml} time: 00:21:57 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024729/ [5024735] rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:11:40 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024735/ [5024737] rados/singleton/{all/resolve_stuck_peering.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:16:14 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024737/ [5024739] rados/cephadm/smoke/{distro/ubuntu_18.04.yaml fixed-2.yaml start.yaml} time: 00:11:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024739/ [5024747] rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{centos_8.yaml} thrashers/minsize_recovery.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} time: 00:25:25 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024747/ [5024749] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04.yaml start.yaml} time: 00:11:12 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024749/ [5024753] rados/singleton/{all/test-crash.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:15:39 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024753/ [5024755] rados/cephadm/workunits/{distro/centos_latest.yaml task/test_cephadm.yaml} time: 00:18:37 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024755/ [5024759] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/on.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_8.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/snaps-few-objects-balanced.yaml} time: 00:27:23 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024759/ [5024761] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml msgr/async-v2only.yaml no_pools.yaml objectstore/bluestore-hybrid.yaml rados.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_clock_with_skews.yaml} time: 00:12:14 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024761/ [5024763] rados/cephadm/with-work/{distro/rhel_8.0.yaml fixed-2.yaml mode/root.yaml msgr/async-v2only.yaml start.yaml tasks/rados_api_tests.yaml} time: 00:37:49 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024763/ [5024765] rados/singleton-nomsgr/{all/health-warnings.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} time: 00:15:04 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024765/ [5024769] rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_8.yaml}} time: 00:19:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024769/ [5024771] rados/cephadm/smoke/{distro/ubuntu_18.04_podman.yaml fixed-2.yaml start.yaml} time: 00:11:44 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024771/ [5024773] rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} tasks/rados_workunit_loadgen_mix.yaml} time: 00:27:59 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024773/ [5024776] rados/standalone/{supported-random-distro$/{centos_8.yaml} workloads/misc.yaml} time: 00:24:56 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024776/ [5024782] rados/verify/{centos_latest.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} time: 00:20:18 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024782/ [5024784] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman.yaml start.yaml} time: 00:11:07 info: http://pulpito.ceph.com/kchai-2020-05-05_13:45:45-rados-wip-kefu-testing-2020-05-05-1333-distro-basic-smithi/5024784/ 2020-05-06T12:29:51.083 INFO:teuthology.results:starting coverage generation 2020-05-06T12:29:51.093 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'