2021-10-18T14:20:44.975 INFO:root:teuthology version: 1.1.0-b35344e8 2021-10-18T14:20:44.976 INFO:teuthology.results:subset: '5346/100000' 2021-10-18T14:20:44.976 INFO:teuthology.results:seed: '7811' 2021-10-19T04:11:28.456 INFO:root:teuthology version: 1.1.0-b35344e8 2021-10-19T04:11:28.495 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-10-19T05:44:38.167 INFO:teuthology.results:Tests finished! gathering results... 2021-10-19T05:45:16.036 INFO:scrape:Found 385 jobs 2021-10-19T05:45:16.575 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447029/teuthology.log 2021-10-19T05:45:16.655 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f7dd17033c0] , 2: gsignal() 2021-10-19T05:45:16.656 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T20:19:57.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive , 2021-10-18T20:19:57.942 INFO:teuthology.task.internal:Transferring archived files... , 2021-10-18T20:19:57.944 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/remote/gibba019 2021-10-19T05:45:16.668 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f7dd17033c0] , 2: gsignal() 2021-10-19T05:45:16.669 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T20:19:57.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive , 2021-10-18T20:19:57.942 INFO:teuthology.task.internal:Transferring archived files... , 2021-10-18T20:19:57.944 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/remote/gibba019 2021-10-19T05:45:16.680 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f7dd17033c0] , 2: gsignal() 2021-10-19T05:45:16.682 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T20:19:57.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive , 2021-10-18T20:19:57.942 INFO:teuthology.task.internal:Transferring archived files... , 2021-10-18T20:19:57.944 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/remote/gibba019 2021-10-19T05:45:16.692 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f7dd17033c0] , 2: gsignal() 2021-10-19T05:45:16.693 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T20:19:57.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive , 2021-10-18T20:19:57.942 INFO:teuthology.task.internal:Transferring archived files... , 2021-10-18T20:19:57.944 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/remote/gibba019 2021-10-19T05:45:16.779 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7fe79d5ecb20] , 2: gsignal() 2021-10-19T05:45:16.780 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T17:55:35.746 DEBUG:teuthology.orchestra.run.gibba005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-18T17:55:35.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-18T17:55:35.806 DEBUG:teuthology.orchestra.run.gibba005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-19T05:45:16.795 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8337-g5db11d39 (5db11d398856f1e5a93048865af3a8da8e128e8d) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7fe79d5ecb20] , 2: gsignal() 2021-10-19T05:45:16.796 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-18T17:55:35.746 DEBUG:teuthology.orchestra.run.gibba005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-18T17:55:35.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-18T17:55:35.806 DEBUG:teuthology.orchestra.run.gibba005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-19T05:45:18.419 INFO:scrape:Found 8 distinct failure reasons 2021-10-19T05:45:18.421 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on gibba008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5db11d398856f1e5a93048865af3a8da8e128e8d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 2021-10-19T05:45:18.421 INFO:scrape:1 jobs: ['6447081'] 2021-10-19T05:45:18.422 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_8.stream}', 'workloads/scrub}'] 2021-10-19T05:45:18.422 INFO:scrape: 2021-10-19T05:45:18.423 INFO:scrape:Dead: None 2021-10-19T05:45:18.423 INFO:scrape:8 jobs: ['6447029', '6447024', '6446946', '6447229', '6446950', '6446944', '6446956', '6447025'] 2021-10-19T05:45:18.423 INFO:scrape: 2021-10-19T05:45:18.424 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*SyntheticMatrixC*/2 --gtest_catch_exceptions=0\'' 2021-10-19T05:45:18.424 INFO:scrape:2 jobs: ['6447057', '6447020'] 2021-10-19T05:45:18.425 INFO:scrape:suites intersection: [] 2021-10-19T05:45:18.425 INFO:scrape:suites union: ['rados/objectstore/{backends/objectstore-bluestore-a', 'rados/objectstore/{backends/objectstore-bluestore-b', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}'] 2021-10-19T05:45:18.426 INFO:scrape: 2021-10-19T05:45:18.426 INFO:scrape:Failure: Command failed on gibba015 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5db11d398856f1e5a93048865af3a8da8e128e8d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5effc88e-3077-11ec-8c28-001a4aab830c -- ceph mon dump -f json' 2021-10-19T05:45:18.427 INFO:scrape:3 jobs: ['6447216', '6447224', '6446995'] 2021-10-19T05:45:18.428 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '3-final}'] 2021-10-19T05:45:18.428 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/client-keyring', '2-services/nfs-ingress', '2-services/nfs-ingress2', '3-final}', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0'] 2021-10-19T05:45:18.429 INFO:scrape: 2021-10-19T05:45:18.430 INFO:scrape:Failure: reached maximum tries (60) after waiting for 900 seconds 2021-10-19T05:45:18.431 INFO:scrape:1 jobs: ['6447036'] 2021-10-19T05:45:18.432 INFO:scrape: 2021-10-19T05:45:18.433 INFO:scrape:Failure: Failed to power on gibba027 2021-10-19T05:45:18.433 INFO:scrape:3 jobs: ['6447237', '6446951', '6446989'] 2021-10-19T05:45:18.434 INFO:scrape: 2021-10-19T05:45:18.435 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on gibba008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5db11d398856f1e5a93048865af3a8da8e128e8d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-10-19T05:45:18.435 INFO:scrape:1 jobs: ['6447222'] 2021-10-19T05:45:18.436 INFO:scrape:suites: ['debug/mgr', 'mon_election/connectivity', 'rados/dashboard/{centos_8.2_container_tools_3.0', 'random-objectstore$/{bluestore-comp-snappy}', 'tasks/e2e}'] 2021-10-19T05:45:18.436 INFO:scrape: 2021-10-19T05:45:18.437 INFO:scrape:Failure: Could not reconnect to ubuntu@gibba008.front.sepia.ceph.com 2021-10-19T05:45:18.437 INFO:scrape:1 jobs: ['6447190'] 2021-10-19T05:45:18.438 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_8.stream}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/cache-agent-big}'] 2021-10-19T05:45:18.439 INFO:scrape: 2021-10-19T05:45:18.456 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/ failed: 8 dead: 12 running: 0 waiting: 0 queued: 0 passed: 365 Fail ================================================================= [6446995] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446995/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446995/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b83df948436d4fe297f820a4d7a808cd Command failed on gibba036 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5db11d398856f1e5a93048865af3a8da8e128e8d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa9f0550-3033-11ec-8c28-001a4aab830c -- ceph mon dump -f json' [6447020] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447020/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447020/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38e9cfc052d446178edd5c2cd84a63e3 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*/2:-*SyntheticMatrixC* --gtest_catch_exceptions=0\'' [6447057] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 01:52:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447057/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cdc25d12186d4a5e8062ada8151bd3e7 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*SyntheticMatrixC*/2 --gtest_catch_exceptions=0\'' [6447081] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/scrub} ----------------------------------------------------------------- time: 01:19:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447081/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447081/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dfec927aedee432b8aa94699673cc304 Command failed (workunit test scrub/osd-scrub-repair.sh) on gibba008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5db11d398856f1e5a93048865af3a8da8e128e8d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- repair.sh' [6447190] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/cache-agent-big} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447190/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447190/ sentry: https://sentry.ceph.com/organizations/ceph/?query=17eadd4a437f49a29cd475d95768a529 Could not reconnect to ubuntu@gibba008.front.sepia.ceph.com [6447216] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447216/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447216/ sentry: https://sentry.ceph.com/organizations/ceph/?query=85608aaa0aa448e2b6e3b94c80241df0 Command failed on gibba015 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5db11d398856f1e5a93048865af3a8da8e128e8d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5effc88e-3077-11ec-8c28-001a4aab830c -- ceph mon dump -f json' [6447222] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447222/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447222/ sentry: https://sentry.ceph.com/organizations/ceph/?query=22670f06b49e469eb93f5d89e2e4441f Command failed (workunit test cephadm/test_dashboard_e2e.sh) on gibba008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5db11d398856f1e5a93048865af3a8da8e128e8d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6447224] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447224/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447224/ sentry: https://sentry.ceph.com/organizations/ceph/?query=175caad767e04edbae0fe2e3532223c8 Command failed on gibba016 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:5db11d398856f1e5a93048865af3a8da8e128e8d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ab289d6a-3079-11ec-8c28-001a4aab830c -- ceph mon dump -f json' Dead ================================================================= [6446944] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446944/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446944/ Error reimaging machines: Failed to power on gibba027 [6446946] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446946/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446946/ Error reimaging machines: Failed to power on gibba027 [6446950] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446950/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446950/ Error reimaging machines: Failed to power on gibba027 [6446951] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446951/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446951/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7b5e1e6de12946c1ae30954ba2867815 Failed to power on gibba027 [6446956] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446956/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446956/ Error reimaging machines: Failed to power on gibba027 [6446989] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446989/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446989/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ccd3cf7c360440cb81cf1abb7385490e Failed to power on gibba027 [6447024] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447024/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447024/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6447025] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447025/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447025/ Error reimaging machines: Failed to power on gibba027 [6447029] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/e2e} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447029/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447029/ Error reimaging machines: Failed to power on gibba027 [6447036] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447036/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447036/ sentry: https://sentry.ceph.com/organizations/ceph/?query=592111295f1947e6a054a499cece646e reached maximum tries (60) after waiting for 900 seconds [6447229] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447229/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447229/ Error reimaging machines: Failed to power on gibba027 [6447237] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447237/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447237/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee94012852744db1af983fc33e94d96c Failed to power on gibba027 Pass ================================================================= [6446935] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446935/ [6446936] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446936/ [6446937] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:27:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446937/ [6446938] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446938/ [6446939] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:42 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446939/ [6446940] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:25:55 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446940/ [6446941] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} tasks/readwrite} time: 00:10:26 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446941/ [6446942] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:35:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446942/ [6446943] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.2_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:11:50 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446943/ [6446945] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream}} time: 00:22:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446945/ [6446947] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446947/ [6446948] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446948/ [6446949] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:09:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446949/ [6446952] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-radosbench} time: 02:09:43 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446952/ [6446953] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446953/ [6446954] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/cache} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446954/ [6446955] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:07:10 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446955/ [6446957] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 01:04:05 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446957/ [6446958] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:33:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446958/ [6446959] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_adoption} time: 00:07:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446959/ [6446960] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/crash} time: 00:17:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446960/ [6446961] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/connectivity} time: 00:26:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446961/ [6446962] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/lockdep} time: 00:11:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446962/ [6446963] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream}} time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446963/ [6446964] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446964/ [6446965] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446965/ [6446966] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446966/ [6446967] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446967/ [6446968] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446968/ [6446969] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446969/ [6446970] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446970/ [6446971] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:38 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446971/ [6446972] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446972/ [6446973] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446973/ [6446974] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream} tasks/mon_clock_no_skews} time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446974/ [6446975] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446975/ [6446976] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} time: 00:55:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446976/ [6446977] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps} time: 00:11:23 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446977/ [6446978] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:34:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446978/ [6446979] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446979/ [6446980] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446980/ [6446981] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} time: 00:38:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446981/ [6446982] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446982/ [6446983] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:08 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446983/ [6446984] rados/singleton/{all/deduptool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446984/ [6446985] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:18:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446985/ [6446986] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446986/ [6446987] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:12:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446987/ [6446988] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/flannel rook/master} time: 00:30:24 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446988/ [6446990] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:23:21 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446990/ [6446991] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:55 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446991/ [6446992] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm_repos} time: 00:05:19 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446992/ [6446993] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446993/ [6446994] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:08:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446994/ [6446996] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446996/ [6446997] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 04:08:11 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446997/ [6446998] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446998/ [6446999] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} tasks/failover} time: 00:13:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6446999/ [6447000] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447000/ [6447001] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447001/ [6447002] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/classic} time: 00:25:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447002/ [6447003] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} time: 00:43:11 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447003/ [6447004] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:25:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447004/ [6447005] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:15:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447005/ [6447006] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream}} time: 00:09:27 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447006/ [6447007] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_nfs} time: 00:19:42 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447007/ [6447008] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447008/ [6447009] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:23:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447009/ [6447010] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447010/ [6447011] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:31:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447011/ [6447012] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:51:34 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447012/ [6447013] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:09:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447013/ [6447014] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:40:23 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447014/ [6447015] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447015/ [6447016] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447016/ [6447017] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447017/ [6447018] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:32:25 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447018/ [6447019] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447019/ [6447021] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447021/ [6447022] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:15:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447022/ [6447023] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:31:45 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447023/ [6447026] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447026/ [6447027] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:07:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447027/ [6447028] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:19:08 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447028/ [6447030] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447030/ [6447031] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447031/ [6447032] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:11:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447032/ [6447033] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:19:27 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447033/ [6447034] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447034/ [6447035] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/insights} time: 00:12:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447035/ [6447037] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447037/ [6447038] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/lockdep} time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447038/ [6447039] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447039/ [6447040] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447040/ [6447041] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:26 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447041/ [6447042] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 03:09:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447042/ [6447043] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:25:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447043/ [6447044] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:11:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447044/ [6447045] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream}} time: 00:07:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447045/ [6447046] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:47:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447046/ [6447047] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:22:24 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447047/ [6447048] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:34 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447048/ [6447049] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447049/ [6447050] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_adoption} time: 00:07:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447050/ [6447051] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:17:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447051/ [6447052] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447052/ [6447053] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/connectivity} time: 00:24:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447053/ [6447054] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 01:25:34 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447054/ [6447055] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:50 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447055/ [6447056] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447056/ [6447058] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447058/ [6447059] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447059/ [6447060] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:24:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447060/ [6447061] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:23:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447061/ [6447062] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447062/ [6447063] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:34 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447063/ [6447064] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:17:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447064/ [6447065] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447065/ [6447066] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:37:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447066/ [6447067] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:16:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447067/ [6447068] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:14:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447068/ [6447069] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:35:51 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447069/ [6447070] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:26:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447070/ [6447071] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:33:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447071/ [6447072] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:28:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447072/ [6447073] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:14:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447073/ [6447074] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447074/ [6447075] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:30:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447075/ [6447076] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447076/ [6447077] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:31:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447077/ [6447078] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:08 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447078/ [6447079] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447079/ [6447080] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447080/ [6447082] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm_repos} time: 00:05:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447082/ [6447083] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/host rook/1.7.0} time: 00:14:43 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447083/ [6447084] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:09:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447084/ [6447085] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:34:25 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447085/ [6447086] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:11:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447086/ [6447087] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:16:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447087/ [6447088] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:26:55 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447088/ [6447089] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:06:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447089/ [6447090] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{centos_8}} time: 00:25:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447090/ [6447091] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447091/ [6447092] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:06:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447092/ [6447093] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447093/ [6447094] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447094/ [6447095] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447095/ [6447096] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447096/ [6447097] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:09:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447097/ [6447098] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/classic} time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447098/ [6447099] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447099/ [6447100] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447100/ [6447101] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:29:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447101/ [6447102] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_nfs} time: 00:19:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447102/ [6447103] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:51:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447103/ [6447104] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects} time: 00:19:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447104/ [6447105] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:06:43 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447105/ [6447106] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:13 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447106/ [6447107] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:38:11 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447107/ [6447108] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:18:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447108/ [6447109] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:20:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447109/ [6447110] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:17:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447110/ [6447111] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:28:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447111/ [6447112] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:25:13 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447112/ [6447113] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447113/ [6447114] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:10:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447114/ [6447115] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:15:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447115/ [6447116] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447116/ [6447117] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_orch_cli} time: 00:11:07 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447117/ [6447118] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447118/ [6447119] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:07:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447119/ [6447120] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447120/ [6447121] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447121/ [6447122] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447122/ [6447123] rados/standalone/{supported-random-distro$/{rhel_8} workloads/c2c} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447123/ [6447124] rados/upgrade/parallel/{0-distro$/{centos_8.3_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:14:45 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447124/ [6447125] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:19:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447125/ [6447126] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447126/ [6447127] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:23:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447127/ [6447128] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447128/ [6447129] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:07:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447129/ [6447130] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:27:31 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447130/ [6447131] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:46:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447131/ [6447132] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:01 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447132/ [6447133] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447133/ [6447134] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr orchestrator_cli} time: 00:20:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447134/ [6447135] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:12:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447135/ [6447136] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream}} time: 00:28:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447136/ [6447137] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:26:25 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447137/ [6447138] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447138/ [6447139] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream}} time: 00:09:19 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447139/ [6447140] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447140/ [6447141] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447141/ [6447142] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:30:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447142/ [6447143] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:10:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447143/ [6447144] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:17:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447144/ [6447145] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.3_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:09:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447145/ [6447146] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447146/ [6447147] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:18:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447147/ [6447148] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:32:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447148/ [6447149] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:58:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447149/ [6447150] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447150/ [6447151] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream} tasks/rados_stress_watch} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447151/ [6447152] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:39:38 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447152/ [6447153] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:27:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447153/ [6447154] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_adoption} time: 00:07:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447154/ [6447155] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:42:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447155/ [6447156] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/connectivity} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447156/ [6447157] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 01:09:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447157/ [6447158] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447158/ [6447159] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447159/ [6447160] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447160/ [6447161] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:09:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447161/ [6447162] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:21 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447162/ [6447163] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447163/ [6447164] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447164/ [6447165] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream}} time: 00:23:27 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447165/ [6447166] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:07:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447166/ [6447167] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447167/ [6447168] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:07:23 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447168/ [6447169] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:17:07 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447169/ [6447170] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447170/ [6447171] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447171/ [6447172] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:44:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447172/ [6447173] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:45:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447173/ [6447174] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447174/ [6447175] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:01 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447175/ [6447176] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447176/ [6447177] rados/singleton/{all/peer mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:07:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447177/ [6447178] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_striper} time: 00:07:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447178/ [6447179] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:14:45 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447179/ [6447180] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:07:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447180/ [6447181] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} time: 00:32:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447181/ [6447182] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm_repos} time: 00:05:13 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447182/ [6447183] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/workunits} time: 00:08:44 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447183/ [6447184] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447184/ [6447185] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:24 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447185/ [6447186] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447186/ [6447187] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:07:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447187/ [6447188] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447188/ [6447189] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:24 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447189/ [6447191] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447191/ [6447192] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/classic} time: 00:24:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447192/ [6447193] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:14:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447193/ [6447194] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447194/ [6447195] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:14:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447195/ [6447196] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:29:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447196/ [6447197] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447197/ [6447198] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:15:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447198/ [6447199] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_nfs} time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447199/ [6447200] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:34 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447200/ [6447201] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:11:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447201/ [6447202] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:22:47 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447202/ [6447203] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:07:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447203/ [6447204] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:31:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447204/ [6447205] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447205/ [6447206] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:28:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447206/ [6447207] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447207/ [6447208] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447208/ [6447209] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:33:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447209/ [6447210] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_workunits} time: 00:35:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447210/ [6447211] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:21 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447211/ [6447212] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:29:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447212/ [6447213] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447213/ [6447214] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:16:36 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447214/ [6447215] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447215/ [6447217] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447217/ [6447218] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:21:25 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447218/ [6447219] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:55 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447219/ [6447220] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_orch_cli} time: 00:11:05 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447220/ [6447221] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447221/ [6447223] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447223/ [6447225] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:23:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447225/ [6447226] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:25:42 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447226/ [6447227] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:31 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447227/ [6447228] rados/singleton/{all/random-eio mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447228/ [6447230] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447230/ [6447231] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:19:38 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447231/ [6447232] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.2_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447232/ [6447233] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:10:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447233/ [6447234] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:20:05 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447234/ [6447235] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:33:59 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447235/ [6447236] rados/singleton/{all/rebuild-mondb mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:19:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447236/ [6447238] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447238/ [6447239] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447239/ [6447240] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447240/ [6447241] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_adoption} time: 00:08:15 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447241/ [6447242] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} tasks/mon_clock_with_skews} time: 00:08:31 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447242/ [6447243] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447243/ [6447244] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447244/ [6447245] rados/singleton/{all/recovery-preemption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:43:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447245/ [6447246] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:18:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447246/ [6447247] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:03 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447247/ [6447248] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/connectivity} time: 00:26:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447248/ [6447249] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} time: 00:24:58 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447249/ [6447250] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:23:26 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447250/ [6447251] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447251/ [6447252] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:36:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447252/ [6447253] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447253/ [6447254] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447254/ [6447255] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:13:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447255/ [6447256] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:22:56 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447256/ [6447257] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/failover} time: 00:20:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447257/ [6447258] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447258/ [6447259] rados/singleton/{all/resolve_stuck_peering mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:07:03 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447259/ [6447260] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447260/ [6447261] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-radosbench} time: 02:13:07 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447261/ [6447262] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:15:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447262/ [6447263] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache} time: 00:10:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447263/ [6447264] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447264/ [6447265] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:25:54 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447265/ [6447266] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:32:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447266/ [6447267] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:23:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447267/ [6447268] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:51 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447268/ [6447269] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447269/ [6447270] rados/singleton/{all/test-crash mon_election/classic msgr-failures/none msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream}} time: 00:09:37 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447270/ [6447271] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:31:39 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447271/ [6447272] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:32:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447272/ [6447273] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447273/ [6447274] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed} time: 00:11:07 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447274/ [6447275] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:17:21 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447275/ [6447276] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/1.7.0} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447276/ [6447277] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:22:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447277/ [6447278] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm_repos} time: 00:05:21 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447278/ [6447279] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447279/ [6447280] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} tasks/mon_recovery} time: 00:18:55 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447280/ [6447281] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447281/ [6447282] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:22 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447282/ [6447283] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:47 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447283/ [6447284] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:05 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447284/ [6447285] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:50:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447285/ [6447286] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/classic} time: 00:25:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447286/ [6447287] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/pool-create-delete} time: 00:15:52 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447287/ [6447288] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/dedup-io-snaps} time: 00:17:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447288/ [6447289] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:11 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447289/ [6447290] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:15:46 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447290/ [6447291] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:11:33 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447291/ [6447292] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:25:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447292/ [6447293] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447293/ [6447294] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447294/ [6447295] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:08:38 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447295/ [6447296] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 01:01:24 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447296/ [6447297] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} time: 00:51:09 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447297/ [6447298] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_nfs} time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447298/ [6447299] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:26:41 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447299/ [6447300] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:08 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447300/ [6447301] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:23 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447301/ [6447302] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:50:04 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447302/ [6447303] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:05:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447303/ [6447304] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:18 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447304/ [6447305] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 01:31:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447305/ [6447306] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:15:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447306/ [6447307] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447307/ [6447308] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:26:17 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447308/ [6447309] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447309/ [6447310] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:15:30 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447310/ [6447311] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:24:07 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447311/ [6447312] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_orch_cli} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447312/ [6447313] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:05 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447313/ [6447314] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:20:16 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447314/ [6447315] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/dashboard} time: 01:08:28 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447315/ [6447316] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:48 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447316/ [6447317] rados/upgrade/parallel/{0-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:24:00 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447317/ [6447318] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream}} time: 00:28:43 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447318/ [6447319] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:06:57 info: http://pulpito.ceph.com/teuthology-2021-10-17_07:01:03-rados-master-distro-default-gibba/6447319/