2021-10-25T10:36:57.469 INFO:root:teuthology version: 1.1.0-c56135d1 2021-10-25T10:36:57.470 INFO:teuthology.results:subset: '1089/100000' 2021-10-25T10:36:57.471 INFO:teuthology.results:seed: '4061' 2021-10-27T06:40:28.299 INFO:root:teuthology version: 1.1.0-c56135d1 2021-10-27T06:40:28.338 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-10-27T09:01:40.015 INFO:teuthology.results:Tests finished! gathering results... 2021-10-27T09:02:14.076 INFO:scrape:Found 405 jobs 2021-10-27T09:02:17.596 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8450-ge7d152fc (e7d152fce8107992eb45c87b4c82b40b6629dd7c) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f63ee8b7b20] , 2: gsignal() 2021-10-27T09:02:17.597 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-27T02:30:23.403 DEBUG:teuthology.orchestra.run.gibba035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-27T02:30:23.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-27T02:30:23.460 DEBUG:teuthology.orchestra.run.gibba035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-27T09:02:17.607 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8450-ge7d152fc (e7d152fce8107992eb45c87b4c82b40b6629dd7c) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f63ee8b7b20] , 2: gsignal() 2021-10-27T09:02:17.608 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-27T02:30:23.403 DEBUG:teuthology.orchestra.run.gibba035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-27T02:30:23.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-27T02:30:23.460 DEBUG:teuthology.orchestra.run.gibba035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-27T09:02:17.617 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8450-ge7d152fc (e7d152fce8107992eb45c87b4c82b40b6629dd7c) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f63ee8b7b20] , 2: gsignal() 2021-10-27T09:02:17.618 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-27T02:30:23.403 DEBUG:teuthology.orchestra.run.gibba035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-27T02:30:23.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-27T02:30:23.460 DEBUG:teuthology.orchestra.run.gibba035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-27T09:02:17.626 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8450-ge7d152fc (e7d152fce8107992eb45c87b4c82b40b6629dd7c) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f63ee8b7b20] , 2: gsignal() 2021-10-27T09:02:17.627 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-27T02:30:23.403 DEBUG:teuthology.orchestra.run.gibba035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-27T02:30:23.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-27T02:30:23.460 DEBUG:teuthology.orchestra.run.gibba035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-27T09:02:17.634 WARNING:scrape:Ignoring malparsed backtrace: ceph version 17.0.0-8450-ge7d152fc (e7d152fce8107992eb45c87b4c82b40b6629dd7c) quincy (dev) , 1: /lib64/libpthread.so.0(+0x12b20) [0x7f63ee8b7b20] , 2: gsignal() 2021-10-27T09:02:17.635 WARNING:scrape:Ignoring malparsed backtrace: 2021-10-27T02:30:23.403 DEBUG:teuthology.orchestra.run.gibba035:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers , 2021-10-27T02:30:23.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump , 2021-10-27T02:30:23.460 DEBUG:teuthology.orchestra.run.gibba035:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-10-27T09:02:18.299 INFO:scrape:Found 11 distinct failure reasons 2021-10-27T09:02:18.300 INFO:scrape:Failure: Test failure: test_ganesha (unittest.loader._FailedTest) 2021-10-27T09:02:18.301 INFO:scrape:2 jobs: ['6459143', '6458939'] 2021-10-27T09:02:18.301 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{centos_8.2_container_tools_3.0', 'tasks/dashboard}'] 2021-10-27T09:02:18.302 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{centos_8.2_container_tools_3.0', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-hybrid}', 'tasks/dashboard}'] 2021-10-27T09:02:18.302 INFO:scrape: 2021-10-27T09:02:18.302 INFO:scrape:Failure: reached maximum tries (100) after waiting for 600 seconds 2021-10-27T09:02:18.303 INFO:scrape:1 jobs: ['6459256'] 2021-10-27T09:02:18.303 INFO:scrape: 2021-10-27T09:02:18.304 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on gibba035 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e7d152fce8107992eb45c87b4c82b40b6629dd7c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-10-27T09:02:18.304 INFO:scrape:2 jobs: ['6459248', '6459044'] 2021-10-27T09:02:18.305 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{centos_8.2_container_tools_3.0', 'tasks/e2e}'] 2021-10-27T09:02:18.305 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{centos_8.2_container_tools_3.0', 'random-objectstore$/{bluestore-comp-zlib}', 'random-objectstore$/{filestore-xfs}', 'tasks/e2e}'] 2021-10-27T09:02:18.305 INFO:scrape: 2021-10-27T09:02:18.306 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2021-10-27T09:02:18.306 INFO:scrape:1 jobs: ['6458955'] 2021-10-27T09:02:18.307 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8.stream}', 'thrashers/default', 'thrashosds-health', 'workloads/rados_api_tests}'] 2021-10-27T09:02:18.307 INFO:scrape: 2021-10-27T09:02:18.307 INFO:scrape:Failure: Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS) 2021-10-27T09:02:18.308 INFO:scrape:4 jobs: ['6459321', '6459221', '6459118', '6459018'] 2021-10-27T09:02:18.308 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0', 'task/test_nfs}'] 2021-10-27T09:02:18.309 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0', 'task/test_nfs}'] 2021-10-27T09:02:18.309 INFO:scrape: 2021-10-27T09:02:18.309 INFO:scrape:Failure: Command failed on gibba017 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*/2:-*SyntheticMatrixC* --gtest_catch_exceptions=0\'' 2021-10-27T09:02:18.310 INFO:scrape:2 jobs: ['6459237', '6459276'] 2021-10-27T09:02:18.310 INFO:scrape:suites intersection: [] 2021-10-27T09:02:18.311 INFO:scrape:suites union: ['rados/objectstore/{backends/objectstore-bluestore-a', 'rados/objectstore/{backends/objectstore-bluestore-b', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}}'] 2021-10-27T09:02:18.311 INFO:scrape: 2021-10-27T09:02:18.311 INFO:scrape:Dead: b'2021-10-27T07:32:13.607 DEBUG:teuthology.exit:Finished running handlers' 2021-10-27T09:02:18.312 INFO:scrape:2 jobs: ['6459204', '6459057'] 2021-10-27T09:02:18.312 INFO:scrape:suites intersection: ['rados'] 2021-10-27T09:02:18.313 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8.stream}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2021-10-27T09:02:18.313 INFO:scrape: 2021-10-27T09:02:18.313 INFO:scrape:Failure: "2021-10-27T01:17:37.524525+0000 mon.b (mon.0) 63 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log 2021-10-27T09:02:18.314 INFO:scrape:1 jobs: ['6459225'] 2021-10-27T09:02:18.314 INFO:scrape:suites: ['clusters/{fixed-4', 'mon_election/connectivity', 'msgr-failures/fastclose', 'objectstore/bluestore-comp-lz4', 'openstack}', 'rados', 'rados/thrash-erasure-code-shec/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashosds-health', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}'] 2021-10-27T09:02:18.315 INFO:scrape: 2021-10-27T09:02:18.315 INFO:scrape:Failure: Found coredumps on ubuntu@gibba043.front.sepia.ceph.com 2021-10-27T09:02:18.316 INFO:scrape:2 jobs: ['6459289', '6459196'] 2021-10-27T09:02:18.316 INFO:scrape:suites intersection: [] 2021-10-27T09:02:18.316 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'debug/mgr', 'mon_election/connectivity', 'objectstore/bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0', 'rados/mgr/{clusters/{2-node-mgr}', 'supported-random-distro$/{centos_8.stream}', 'tasks/progress}'] 2021-10-27T09:02:18.317 INFO:scrape: 2021-10-27T09:02:18.317 INFO:scrape:Failure: Command failed on gibba011 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:e7d152fce8107992eb45c87b4c82b40b6629dd7c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f04c11a-36b6-11ec-8c28-001a4aab830c -- ceph mon dump -f json' 2021-10-27T09:02:18.317 INFO:scrape:2 jobs: ['6459260', '6458959'] 2021-10-27T09:02:18.318 INFO:scrape:suites intersection: ['distro/centos_8.2_container_tools_3.0', 'fixed-2', 'rados/cephadm/smoke/{0-nvme-loop', 'start}'] 2021-10-27T09:02:18.318 INFO:scrape:suites union: ['distro/centos_8.2_container_tools_3.0', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/smoke/{0-nvme-loop', 'start}'] 2021-10-27T09:02:18.319 INFO:scrape: 2021-10-27T09:02:18.319 INFO:scrape:Dead: b'2021-10-27T06:15:30.097 INFO:teuthology.lock.ops:waiting for more gibba machines to be free (need 0 + 4, have 0)...' 2021-10-27T09:02:18.319 INFO:scrape:1 jobs: ['6459185'] 2021-10-27T09:02:18.320 INFO:scrape: 2021-10-27T09:02:18.331 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/ failed: 16 dead: 4 running: 0 waiting: 0 queued: 0 passed: 385 Fail ================================================================= [6458939] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} ----------------------------------------------------------------- time: 00:20:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458939/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458939/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7a1ef7cc98d54debb094333c39119d4c Test failure: test_ganesha (unittest.loader._FailedTest) [6458955] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:27:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458955/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458955/ sentry: https://sentry.ceph.com/organizations/ceph/?query=674e463384b44b2cb2471ccb23fb16d8 Command failed (workunit test rados/test.sh) on gibba037 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e7d152fce8107992eb45c87b4c82b40b6629dd7c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [6458959] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458959/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458959/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b13b4dd2154747698c8abc884840e36e Command failed on gibba044 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:e7d152fce8107992eb45c87b4c82b40b6629dd7c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 073de1a0-3598-11ec-8c28-001a4aab830c -- ceph mon dump -f json' [6459018] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:19:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459018/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459018/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f83a19ed0eb48b4ae708730c8e3b96b Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS) [6459044] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} tasks/e2e} ----------------------------------------------------------------- time: 00:15:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459044/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459044/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2b89fa2651d45bc856f234ea2f0c659 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on gibba023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e7d152fce8107992eb45c87b4c82b40b6629dd7c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6459118] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:19:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459118/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459118/ sentry: https://sentry.ceph.com/organizations/ceph/?query=39decca084ea4cac8e4382b146d605d5 Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS) [6459143] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} ----------------------------------------------------------------- time: 00:21:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459143/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459143/ sentry: https://sentry.ceph.com/organizations/ceph/?query=39be1385f8db4c8a92dbc8f98ea18ea7 Test failure: test_ganesha (unittest.loader._FailedTest) [6459196] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/progress} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459196/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459196/ sentry: https://sentry.ceph.com/organizations/ceph/?query=95c1c5116c284dd99e5e6251aa72e8b0 Could not reconnect to ubuntu@gibba036.front.sepia.ceph.com [6459221] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:19:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459221/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459221/ sentry: https://sentry.ceph.com/organizations/ceph/?query=034ef57b6d2d448d897970a7765da17f Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS) [6459225] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459225/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459225/ "2021-10-27T01:17:37.524525+0000 mon.b (mon.0) 63 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log [6459237] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 02:17:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459237/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459237/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7abf7f5f8dfa4f8094972e563c2fbff0 Command failed on gibba017 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to- stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug- bluestore 20" ceph_test_objectstore --gtest_filter=*/2:-*SyntheticMatrixC* --gtest_catch_exceptions=0\'' [6459248] rados/dashboard/{centos_8.2_container_tools_3.0 debug/mgr mon_election/classic random-objectstore$/{filestore-xfs} tasks/e2e} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459248/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459248/ sentry: https://sentry.ceph.com/organizations/ceph/?query=def1fb7cd36d4e87a039235d169e27ba Command failed (workunit test cephadm/test_dashboard_e2e.sh) on gibba035 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e7d152fce8107992eb45c87b4c82b40b6629dd7c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [6459260] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:07:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459260/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459260/ sentry: https://sentry.ceph.com/organizations/ceph/?query=178726146a69486ba63d9a0f76c39397 Command failed on gibba011 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:e7d152fce8107992eb45c87b4c82b40b6629dd7c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f04c11a-36b6-11ec-8c28-001a4aab830c -- ceph mon dump -f json' [6459276] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:28:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459276/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459276/ sentry: https://sentry.ceph.com/organizations/ceph/?query=180df8d1365a4c8e8828c24ad6405053 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*SyntheticMatrixC*/2 --gtest_catch_exceptions=0\'' [6459289] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:23:46 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459289/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459289/ Found coredumps on ubuntu@gibba043.front.sepia.ceph.com [6459321] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 00:19:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459321/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459321/ sentry: https://sentry.ceph.com/organizations/ceph/?query=75069d96d44b4462b17bc184e46c8639 Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS) Dead ================================================================= [6459057] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459057/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459057/ [6459185] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459185/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459185/ [6459204] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459204/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459204/ [6459256] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459256/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459256/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14a0ce1c8b27433ebdbceeb10b0b0bc3 reached maximum tries (100) after waiting for 600 seconds Pass ================================================================= [6458920] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458920/ [6458921] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:36:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458921/ [6458922] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458922/ [6458923] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:29:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458923/ [6458924] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/progress} time: 00:27:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458924/ [6458925] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:36:54 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458925/ [6458926] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:13:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458926/ [6458927] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:31:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458927/ [6458928] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-snaps} time: 00:10:13 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458928/ [6458929] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final} time: 00:14:39 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458929/ [6458930] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:11:13 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458930/ [6458931] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458931/ [6458932] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458932/ [6458933] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/1-node k8s/1.21 net/flannel rook/1.7.0} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458933/ [6458934] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:16:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458934/ [6458935] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458935/ [6458936] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_orch_cli} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458936/ [6458937] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:27:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458937/ [6458938] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects} time: 00:34:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458938/ [6458940] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458940/ [6458941] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458941/ [6458942] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458942/ [6458943] rados/standalone/{supported-random-distro$/{rhel_8} workloads/c2c} time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458943/ [6458944] rados/upgrade/parallel/{0-distro$/{centos_8.3_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:15:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458944/ [6458945] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458945/ [6458946] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458946/ [6458947] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458947/ [6458948] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458948/ [6458949] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:18:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458949/ [6458950] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:26:34 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458950/ [6458951] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_mon_workunits} time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458951/ [6458952] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458952/ [6458953] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.2_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458953/ [6458954] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458954/ [6458956] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:11:26 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458956/ [6458957] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 01:39:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458957/ [6458958] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:33:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458958/ [6458960] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458960/ [6458961] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458961/ [6458962] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.2_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:09:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458962/ [6458963] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458963/ [6458964] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458964/ [6458965] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458965/ [6458966] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:13 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458966/ [6458967] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8.stream} tasks/prometheus} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458967/ [6458968] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:23:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458968/ [6458969] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:34:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458969/ [6458970] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458970/ [6458971] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458971/ [6458972] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:21:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458972/ [6458973] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_adoption} time: 00:08:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458973/ [6458974] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458974/ [6458975] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:11:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458975/ [6458976] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458976/ [6458977] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:31:27 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458977/ [6458978] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream}} time: 00:28:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458978/ [6458979] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458979/ [6458980] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream}} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458980/ [6458981] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8.stream}} time: 00:13:46 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458981/ [6458982] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:36:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458982/ [6458983] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458983/ [6458984] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458984/ [6458985] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:07:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458985/ [6458986] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:49:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458986/ [6458987] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458987/ [6458988] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:26:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458988/ [6458989] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:08:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458989/ [6458990] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458990/ [6458991] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream} tasks/libcephsqlite} time: 00:16:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458991/ [6458992] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync workloads/snaps-few-objects} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458992/ [6458993] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:20:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458993/ [6458994] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458994/ [6458995] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458995/ [6458996] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/redirect} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458996/ [6458997] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458997/ [6458998] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:19:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458998/ [6458999] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:31 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6458999/ [6459000] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream}} time: 01:26:18 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459000/ [6459001] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm_repos} time: 00:05:46 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459001/ [6459002] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459002/ [6459003] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:19:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459003/ [6459004] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:06:45 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459004/ [6459005] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:25:27 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459005/ [6459006] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 01:48:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459006/ [6459007] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459007/ [6459008] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:37 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459008/ [6459009] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:55:54 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459009/ [6459010] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459010/ [6459011] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459011/ [6459012] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:35:18 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459012/ [6459013] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459013/ [6459014] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:18:49 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459014/ [6459015] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:16:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459015/ [6459016] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8.stream}} time: 00:08:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459016/ [6459017] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:13 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459017/ [6459019] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_set_object} time: 00:19:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459019/ [6459020] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} tasks/rados_api_tests} time: 00:25:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459020/ [6459021] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:19:24 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459021/ [6459022] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:08:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459022/ [6459023] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459023/ [6459024] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:23:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459024/ [6459025] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:08:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459025/ [6459026] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459026/ [6459027] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459027/ [6459028] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete} time: 00:16:54 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459028/ [6459029] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:38:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459029/ [6459030] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:25:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459030/ [6459031] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459031/ [6459032] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 01:20:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459032/ [6459033] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/set-chunks-read} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459033/ [6459034] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459034/ [6459035] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459035/ [6459036] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/host rook/master} time: 00:30:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459036/ [6459037] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:15:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459037/ [6459038] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:14:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459038/ [6459039] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_orch_cli} time: 00:11:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459039/ [6459040] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:08:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459040/ [6459041] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/crash} time: 00:10:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459041/ [6459042] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:39 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459042/ [6459043] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459043/ [6459045] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459045/ [6459046] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:07:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459046/ [6459047] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:26:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459047/ [6459048] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} time: 00:27:54 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459048/ [6459049] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459049/ [6459050] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459050/ [6459051] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:22:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459051/ [6459052] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459052/ [6459053] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:16:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459053/ [6459054] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream}} time: 00:08:24 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459054/ [6459055] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:20:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459055/ [6459056] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:10:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459056/ [6459058] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459058/ [6459059] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} time: 00:28:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459059/ [6459060] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:14:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459060/ [6459061] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:33:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459061/ [6459062] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:47:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459062/ [6459063] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:28:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459063/ [6459064] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459064/ [6459065] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459065/ [6459066] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_adoption} time: 00:08:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459066/ [6459067] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459067/ [6459068] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_5925} time: 00:06:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459068/ [6459069] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:11:22 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459069/ [6459070] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects} time: 00:18:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459070/ [6459071] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459071/ [6459072] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:34:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459072/ [6459073] rados/singleton/{all/deduptool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:06:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459073/ [6459074] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:11:24 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459074/ [6459075] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:35:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459075/ [6459076] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459076/ [6459077] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:15:55 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459077/ [6459078] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/mgr} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459078/ [6459079] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459079/ [6459080] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm} time: 00:13:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459080/ [6459081] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/failover} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459081/ [6459082] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} tasks/rados_python} time: 00:13:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459082/ [6459083] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:49 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459083/ [6459084] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459084/ [6459085] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459085/ [6459086] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459086/ [6459087] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:31:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459087/ [6459088] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:15 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459088/ [6459089] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 01:00:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459089/ [6459090] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:32:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459090/ [6459091] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 01:02:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459091/ [6459092] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} time: 02:23:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459092/ [6459093] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:29:45 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459093/ [6459094] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:20:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459094/ [6459095] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:41:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459095/ [6459096] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final} time: 00:15:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459096/ [6459097] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:07:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459097/ [6459098] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459098/ [6459099] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459099/ [6459100] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:18:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459100/ [6459101] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459101/ [6459102] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459102/ [6459103] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm_repos} time: 00:05:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459103/ [6459104] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream}} time: 00:10:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459104/ [6459105] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:26:49 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459105/ [6459106] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459106/ [6459107] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459107/ [6459108] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459108/ [6459109] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_api_tests} time: 00:33:38 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459109/ [6459110] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:29:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459110/ [6459111] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:06:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459111/ [6459112] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:16:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459112/ [6459113] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459113/ [6459114] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459114/ [6459115] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 01:36:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459115/ [6459116] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459116/ [6459117] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/insights} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459117/ [6459119] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 01:02:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459119/ [6459120] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} time: 00:58:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459120/ [6459121] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459121/ [6459122] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459122/ [6459123] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:47:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459123/ [6459124] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459124/ [6459125] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:22:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459125/ [6459126] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:18:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459126/ [6459127] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:06:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459127/ [6459128] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 01:38:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459128/ [6459129] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459129/ [6459130] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459130/ [6459131] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:33:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459131/ [6459132] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:18:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459132/ [6459133] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459133/ [6459134] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8.stream}} time: 00:11:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459134/ [6459135] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:15:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459135/ [6459136] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/1.7.0} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459136/ [6459137] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:34:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459137/ [6459138] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:14:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459138/ [6459139] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_orch_cli} time: 00:11:18 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459139/ [6459140] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:11:05 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459140/ [6459141] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:26:26 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459141/ [6459142] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:06:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459142/ [6459144] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459144/ [6459145] rados/upgrade/parallel/{0-distro$/{centos_8.2_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:15:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459145/ [6459146] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459146/ [6459147] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:16:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459147/ [6459148] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459148/ [6459149] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459149/ [6459150] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:42:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459150/ [6459151] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459151/ [6459152] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:16:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459152/ [6459153] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:36:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459153/ [6459154] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459154/ [6459155] rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} time: 00:20:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459155/ [6459156] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459156/ [6459157] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459157/ [6459158] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:29:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459158/ [6459159] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:11:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459159/ [6459160] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 02:03:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459160/ [6459161] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.2_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:26 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459161/ [6459162] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:39:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459162/ [6459163] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459163/ [6459164] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:17:37 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459164/ [6459165] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459165/ [6459166] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:12:55 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459166/ [6459167] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:40:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459167/ [6459168] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:28:15 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459168/ [6459169] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459169/ [6459170] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:00:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459170/ [6459171] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:15:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459171/ [6459172] rados/cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:31:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459172/ [6459173] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459173/ [6459174] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_adoption} time: 00:08:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459174/ [6459175] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream}} time: 00:28:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459175/ [6459176] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:31:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459176/ [6459177] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459177/ [6459178] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:32:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459178/ [6459179] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459179/ [6459180] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459180/ [6459181] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:19:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459181/ [6459182] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459182/ [6459183] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459183/ [6459184] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:37 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459184/ [6459186] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:17:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459186/ [6459187] rados/multimon/{clusters/6 mon_election/classic msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459187/ [6459188] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:27:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459188/ [6459189] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:20:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459189/ [6459190] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:04:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459190/ [6459191] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:33:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459191/ [6459192] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:18:55 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459192/ [6459193] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits} time: 00:23:05 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459193/ [6459194] rados/cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:20:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459194/ [6459195] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459195/ [6459197] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459197/ [6459198] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:19:26 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459198/ [6459199] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8.stream}} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459199/ [6459200] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:16:57 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459200/ [6459201] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459201/ [6459202] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:20:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459202/ [6459203] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459203/ [6459205] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:25:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459205/ [6459206] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm_repos} time: 00:05:22 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459206/ [6459207] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:29:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459207/ [6459208] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:10:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459208/ [6459209] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:26:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459209/ [6459210] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459210/ [6459211] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:20:11 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459211/ [6459212] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:16:27 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459212/ [6459213] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:05:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459213/ [6459214] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 04:11:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459214/ [6459215] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:20:22 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459215/ [6459216] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459216/ [6459217] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459217/ [6459218] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:27:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459218/ [6459219] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:28 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459219/ [6459220] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:16:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459220/ [6459222] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459222/ [6459223] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:08:05 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459223/ [6459224] rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:23:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459224/ [6459226] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:14:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459226/ [6459227] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:31:31 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459227/ [6459228] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459228/ [6459229] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:40:54 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459229/ [6459230] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/snaps-few-objects} time: 00:25:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459230/ [6459231] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:26 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459231/ [6459232] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:11:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459232/ [6459233] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:07:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459233/ [6459234] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:18:33 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459234/ [6459235] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:12:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459235/ [6459236] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:31:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459236/ [6459238] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/flannel rook/master} time: 00:26:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459238/ [6459239] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:15:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459239/ [6459240] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:07:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459240/ [6459241] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:11:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459241/ [6459242] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_orch_cli} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459242/ [6459243] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8.stream}} time: 00:57:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459243/ [6459244] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:44 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459244/ [6459245] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:41 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459245/ [6459246] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:14 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459246/ [6459247] rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_latest} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} time: 00:52:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459247/ [6459249] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459249/ [6459250] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind} time: 00:42:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459250/ [6459251] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls mon_election/classic} time: 00:26:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459251/ [6459252] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw 3-final} time: 00:15:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459252/ [6459253] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:21:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459253/ [6459254] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:25:21 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459254/ [6459255] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:10:59 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459255/ [6459257] rados/cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:11:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459257/ [6459258] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:16:51 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459258/ [6459259] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/osd} time: 03:04:31 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459259/ [6459261] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:35:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459261/ [6459262] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:11:53 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459262/ [6459263] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:40:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459263/ [6459264] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459264/ [6459265] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:47:31 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459265/ [6459266] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:40:05 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459266/ [6459267] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459267/ [6459268] rados/multimon/{clusters/21 mon_election/classic msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459268/ [6459269] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:28:32 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459269/ [6459270] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:27 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459270/ [6459271] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:23 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459271/ [6459272] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:27:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459272/ [6459273] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_adoption} time: 00:07:56 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459273/ [6459274] rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/snaps-few-objects} time: 00:44:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459274/ [6459275] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/workunits} time: 00:08:06 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459275/ [6459277] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459277/ [6459278] rados/singleton/{all/peer mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:06:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459278/ [6459279] rados/cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459279/ [6459280] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459280/ [6459281] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.3_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459281/ [6459282] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459282/ [6459283] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/classic task/test_cephadm} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459283/ [6459284] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459284/ [6459285] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:20:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459285/ [6459286] rados/cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:13:16 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459286/ [6459287] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:08:50 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459287/ [6459288] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/lockdep} time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459288/ [6459290] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:24:19 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459290/ [6459291] rados/cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:28:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459291/ [6459292] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:18:09 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459292/ [6459293] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:06:07 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459293/ [6459294] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:34:12 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459294/ [6459295] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459295/ [6459296] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:09:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459296/ [6459297] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459297/ [6459298] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:16:00 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459298/ [6459299] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:16:08 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459299/ [6459300] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459300/ [6459301] rados/standalone/{supported-random-distro$/{centos_8.stream} workloads/scrub} time: 02:31:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459301/ [6459302] rados/cephadm/workunits/{0-distro/centos_8.2_container_tools_3.0 mon_election/connectivity task/test_cephadm_repos} time: 00:10:29 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459302/ [6459303] rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_latest} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:21:34 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459303/ [6459304] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:30 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459304/ [6459305] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459305/ [6459306] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8.stream} thrashers/none thrashosds-health workloads/radosbench} time: 01:22:22 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459306/ [6459307] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8.stream}} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459307/ [6459308] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:07:43 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459308/ [6459309] rados/cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:15 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459309/ [6459310] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{rhel_8}} time: 00:36:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459310/ [6459311] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:24:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459311/ [6459312] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8.stream} tasks/crash} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459312/ [6459313] rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8.stream} thrashers/one workloads/pool-create-delete} time: 00:17:47 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459313/ [6459314] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459314/ [6459315] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459315/ [6459316] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:42 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459316/ [6459317] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8.stream}} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459317/ [6459318] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:18:36 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459318/ [6459319] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:14:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459319/ [6459320] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:16:22 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459320/ [6459322] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:41:10 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459322/ [6459323] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:35 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459323/ [6459324] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:58 info: http://pulpito.ceph.com/teuthology-2021-10-24_07:01:02-rados-master-distro-default-gibba/6459324/