2023-11-15T21:12:44.200 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-15T21:12:44.200 INFO:teuthology.results:subset: '111/120000'
2023-11-15T21:12:44.200 INFO:teuthology.results:seed: '7739'
2023-11-16T01:57:52.410 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-16T01:57:52.912 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-16T05:00:11.699 INFO:teuthology.results:Tests finished! gathering results...
2023-11-16T05:00:44.596 INFO:scrape:Found 427 jobs
2023-11-16T05:00:51.983 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:57.994 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.048 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.048 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.074 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.075 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.112 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.113 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.141 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.142 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.171 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.172 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.197 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.197 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.222 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.223 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.246 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.246 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.270 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.271 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.295 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.296 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.322 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.323 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.350 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.351 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.377 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.378 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.401 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.401 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:00:58.430 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES:  ceph version 16.2.14-171-ged3c8e92 (ed3c8e926b9de93026c8a7c1d86be5d6821b52ac) pacific (stable)
,  1: /lib64/libpthread.so.0(+0x12cf0) [0x7f51e52c2cf0]
,  2: pthread_getname_np()

2023-11-16T05:00:58.431 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace!
2023-11-16T05:01:06.703 INFO:scrape:Found 15 distinct failure reasons
2023-11-16T05:01:06.704 INFO:scrape:Failure: Command failed on smithi204 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ed3c8e926b9de93026c8a7c1d86be5d6821b52ac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4454796e-8424-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json'
2023-11-16T05:01:06.704 INFO:scrape:2 jobs: ['7459726', '7459661']
2023-11-16T05:01:06.704 INFO:scrape:suites intersection: ['1-start']
2023-11-16T05:01:06.704 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/repave-all}', '2-thrash', '3-tasks/radosbench', 'fixed-2', 'msgr/async-v1only', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'root}']
2023-11-16T05:01:06.704 INFO:scrape:
2023-11-16T05:01:06.704 INFO:scrape:Failure: reached maximum tries (301) after waiting for 300 seconds
2023-11-16T05:01:06.704 INFO:scrape:5 jobs: ['7459748', '7459760', '7459637', '7459714', '7459739']
2023-11-16T05:01:06.705 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '3-final}']
2023-11-16T05:01:06.705 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/rgw-ingress', '3-final}', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04']
2023-11-16T05:01:06.705 INFO:scrape:
2023-11-16T05:01:06.705 INFO:scrape:Failure: Command failed on smithi145 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1d48eba4-8404-11ee-8dbe-212e2dc638e7 -e sha1=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2023-11-16T05:01:06.705 INFO:scrape:3 jobs: ['7459415', '7459545', '7459675']
2023-11-16T05:01:06.705 INFO:scrape:suites intersection: []
2023-11-16T05:01:06.705 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-final}', '4-wait', '5-upgrade-ls', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04']
2023-11-16T05:01:06.705 INFO:scrape:
2023-11-16T05:01:06.706 INFO:scrape:Failure: Failed to reconnect to smithi119
2023-11-16T05:01:06.706 INFO:scrape:10 jobs: ['7459667', '7459737', '7459616', '7459422', '7459587', '7459471', '7459544', '7459433', '7459712', '7459469']
2023-11-16T05:01:06.706 INFO:scrape:suites intersection: []
2023-11-16T05:01:06.706 INFO:scrape:suites union: ['0-nvme-loop', '1-install/luminous', '1-install/mimic', '1-install/nautilus', '1-install/nautilus-v1only', '1-install/octopus', '1-start', '2-node-mgr', '2-ops/rm-zap-flag}', '2-services/nfs-ingress2', '3-final}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'distro/ubuntu_18.04', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'orchestrator_cli}', 'rados', 'rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_18.04}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'start}', 'thrashers/default', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/rbd_cls}', 'workloads/test_rbd_api}']
2023-11-16T05:01:06.706 INFO:scrape:
2023-11-16T05:01:06.707 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
2023-11-16T05:01:06.707 INFO:scrape:2 jobs: ['7459503', '7459716']
2023-11-16T05:01:06.707 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}']
2023-11-16T05:01:06.707 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}']
2023-11-16T05:01:06.708 INFO:scrape:
2023-11-16T05:01:06.708 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)
2023-11-16T05:01:06.708 INFO:scrape:4 jobs: ['7459359', '7459570', '7459467', '7459680']
2023-11-16T05:01:06.708 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_nfs}']
2023-11-16T05:01:06.708 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_nfs}']
2023-11-16T05:01:06.708 INFO:scrape:
2023-11-16T05:01:06.708 INFO:scrape:Failure: "2023-11-15T22:30:50.245514+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check failed: 2 client(s) laggy due to laggy OSDs (MDS_CLIENTS_LAGGY)" in cluster log
2023-11-16T05:01:06.708 INFO:scrape:4 jobs: ['7459387', '7459734', '7459382', '7459478']
2023-11-16T05:01:06.709 INFO:scrape:suites intersection: ['mon_election/classic']
2023-11-16T05:01:06.709 INFO:scrape:suites union: ['ceph', 'clusters/{2-node-mgr}', 'clusters/{fixed-2', 'd-thrash/none', 'debug/mgr', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/dashboard/{centos_8.stream_container_tools', 'rados/verify/{centos_latest', 'random-objectstore$/{bluestore-comp-lz4}', 'supported-random-distro$/{rhel_8}', 'tasks/dashboard}', 'tasks/rados_cls_all', 'tasks/rados_cls_all}', 'validater/lockdep}']
2023-11-16T05:01:06.709 INFO:scrape:
2023-11-16T05:01:06.709 INFO:scrape:Failure: Command failed on smithi195 with status 5: 'sudo systemctl stop ceph-5014dc1c-841f-11ee-8dbe-212e2dc638e7@mon.smithi195'
2023-11-16T05:01:06.709 INFO:scrape:9 jobs: ['7459649', '7459719', '7459611', '7459749', '7459589', '7459703', '7459686', '7459565', '7459672']
2023-11-16T05:01:06.709 INFO:scrape:suites intersection: ['3-upgrade-with-workload']
2023-11-16T05:01:06.709 INFO:scrape:suites union: ['1-bootstrap/16.2.5', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'roles', 'tasks/{0-from/v16.2.4']
2023-11-16T05:01:06.710 INFO:scrape:
2023-11-16T05:01:06.710 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2023-11-16T05:01:06.710 INFO:scrape:2 jobs: ['7459609', '7459394']
2023-11-16T05:01:06.710 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}']
2023-11-16T05:01:06.711 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}']
2023-11-16T05:01:06.711 INFO:scrape:
2023-11-16T05:01:06.711 INFO:scrape:Failure: Command failed on smithi005 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ed3c8e926b9de93026c8a7c1d86be5d6821b52ac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 515621f4-8414-11ee-8dbe-212e2dc638e7 -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\''
2023-11-16T05:01:06.711 INFO:scrape:1 jobs: ['7459529']
2023-11-16T05:01:06.711 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04']
2023-11-16T05:01:06.712 INFO:scrape:
2023-11-16T05:01:06.712 INFO:scrape:Failure: Command failed (workunit test mon/crush_ops.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh'
2023-11-16T05:01:06.712 INFO:scrape:1 jobs: ['7459694']
2023-11-16T05:01:06.712 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/classic', 'msgr-failures/mon-delay', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/one', 'workloads/rados_mon_workunits}']
2023-11-16T05:01:06.712 INFO:scrape:
2023-11-16T05:01:06.712 INFO:scrape:Assertion: 8: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f51da87d605]
2023-11-16T05:01:06.713 INFO:scrape:1 jobs: ['7459340']
2023-11-16T05:01:06.713 INFO:scrape:suites: ['rados/objectstore/{backends/objectstore', 'supported-random-distro$/{centos_8}}']
2023-11-16T05:01:06.713 INFO:scrape:
2023-11-16T05:01:06.713 INFO:scrape:Failure: timeout expired in wait_until_healthy
2023-11-16T05:01:06.713 INFO:scrape:3 jobs: ['7459629', '7459756', '7459613']
2023-11-16T05:01:06.714 INFO:scrape:suites intersection: []
2023-11-16T05:01:06.714 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '2-repo_digest/defaut', '3-upgrade-with-workload', '3-upgrade/simple', '4-final}', '4-wait', '5-upgrade-ls', 'mon_election/classic}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9']
2023-11-16T05:01:06.714 INFO:scrape:
2023-11-16T05:01:06.714 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi072 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2023-11-16T05:01:06.714 INFO:scrape:2 jobs: ['7459744', '7459640']
2023-11-16T05:01:06.714 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2023-11-16T05:01:06.714 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2023-11-16T05:01:06.714 INFO:scrape:
2023-11-16T05:01:06.715 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi129 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'
2023-11-16T05:01:06.715 INFO:scrape:1 jobs: ['7459474']
2023-11-16T05:01:06.715 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/none', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb', 'supported-random-distro$/{rhel_8}}']
2023-11-16T05:01:06.715 INFO:scrape:
2023-11-16T05:01:06.731 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/
failed:  50
dead:    0
running: 0
waiting: 0
queued:  0
passed:  377


Fail
=================================================================
[7459340]  rados/objectstore/{backends/objectstore supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   05:19:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459340/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459340/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cc13a50435d2414ba468ad72058dd5ae

    Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir
    $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384
    && CEPH_ARGS="--no-log-to-stderr --log-file
    $TESTDIR/archive/ceph_test_objectstore.log --debug-filestore 20 --debug-
    bluestore 20" ceph_test_objectstore --gtest_filter=-*/3
    --gtest_catch_exceptions=0\''

[7459359]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:37:23
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459359/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459359/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d3e6bd9437c542c8a44fbbe1b79385da

    Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)

[7459382]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_cls_all}
-----------------------------------------------------------------
time:   00:20:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459382/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459382/

    "2023-11-15T22:09:04.973403+0000 mon.a (mon.0) 525 : cluster [WRN] Health
    check failed: 1 pool(s) do not have an application enabled
    (POOL_APP_NOT_ENABLED)" in cluster log

[7459387]  rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/dashboard}
-----------------------------------------------------------------
time:   01:06:11
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459387/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459387/

    "2023-11-15T22:30:50.245514+0000 mon.a (mon.0) 3128 : cluster [WRN] Health
    check failed: 2 client(s) laggy due to laggy OSDs (MDS_CLIENTS_LAGGY)" in
    cluster log

[7459394]  rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e}
-----------------------------------------------------------------
time:   00:25:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459394/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459394/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fe1c0d75fdf545068068fbe918dbfc18

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi031
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7459415]  rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic}
-----------------------------------------------------------------
time:   00:20:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459415/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459415/
sentry: https://sentry.ceph.com/organizations/ceph/?query=201bb325075a408b9aaf59ee38de95b9

    Command failed on smithi145 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    1d48eba4-8404-11ee-8dbe-212e2dc638e7 -e
    sha1=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7459422]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459422/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459422/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fb357fcdb2314ac594d91c13c90644fc

    Failed to reconnect to smithi099

[7459433]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459433/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459433/
sentry: https://sentry.ceph.com/organizations/ceph/?query=44482607bf584b88ac8e3cb9cf88fad0

    Failed to reconnect to smithi002

[7459467]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:29:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459467/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459467/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fafbac84f65c409aa55a760c1fefb706

    Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)

[7459469]  rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459469/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459469/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9cf18fcfbfcc494a8f3b8fe19ae756a4

    Failed to reconnect to smithi119

[7459471]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459471/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459471/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1b077ea979654233af3b2f7896a6390e

    Failed to reconnect to smithi102

[7459474]  rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:16:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459474/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459474/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e036dafbcd5a4985a75853de6ca7762b

    Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on
    smithi129 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

[7459478]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep}
-----------------------------------------------------------------
time:   00:24:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459478/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459478/

    "2023-11-16T00:04:51.058058+0000 mon.a (mon.0) 475 : cluster [WRN] Health
    check failed: 1 pool(s) do not have an application enabled
    (POOL_APP_NOT_ENABLED)" in cluster log

[7459503]  rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e}
-----------------------------------------------------------------
time:   00:07:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459503/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459503/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a66226fe0627470993d50613d5822b01

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1
    =ed3c8e926b9de93026c8a7c1d86be5d6821b52ac

[7459529]  rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:16:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459529/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459529/
sentry: https://sentry.ceph.com/organizations/ceph/?query=627b8aaf0f8f47ccb586d62e2429514f

    Command failed on smithi005 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:ed3c8e926b9de93026c8a7c1d86be5d6821b52ac shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    515621f4-8414-11ee-8dbe-212e2dc638e7 -- bash -c \'set -e\nset -x\nceph orch
    ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk
    \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk
    \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk
    \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid
    $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do
    sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon
    add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep
    5 ; done\n\''

[7459544]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459544/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459544/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a919df129094430a0be5f68cdf0416a

    Failed to reconnect to smithi183

[7459545]  rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:20:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459545/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f6de1bae210a4d72933d8724e2fbcd19

    Command failed on smithi023 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    5dba856a-841a-11ee-8dbe-212e2dc638e7 -e
    sha1=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7459565]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:25:40
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459565/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459565/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1cb1a32a8aa04e59b035a1eae4fcd7e9

    Command failed on smithi186 with status 5: 'sudo systemctl stop
    ceph-24648250-841c-11ee-8dbe-212e2dc638e7@mon.smithi186'

[7459570]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:27:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459570/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459570/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69a4c3b504b744f29d53a739b7bb5f91

    Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)

[7459587]  rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459587/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459587/
sentry: https://sentry.ceph.com/organizations/ceph/?query=80f360a2855244c9bfea708f4de83e83

    Failed to reconnect to smithi154

[7459589]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:27:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459589/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459589/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aee679322b184635b7356a1df98f3eea

    Command failed on smithi091 with status 5: 'sudo systemctl stop
    ceph-6d5aeb66-841c-11ee-8dbe-212e2dc638e7@mon.smithi091'

[7459609]  rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e}
-----------------------------------------------------------------
time:   00:27:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459609/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459609/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c4ca0dfad177499793b93cabd3b14342

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi090
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7459611]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:24:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459611/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459611/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2109b3ae4d0244bdad2158692905393b

    Command failed on smithi134 with status 5: 'sudo systemctl stop
    ceph-7905ec40-841c-11ee-8dbe-212e2dc638e7@mon.smithi134'

[7459613]  rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:25:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459613/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459613/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f625c8da377943328df89379593fe235

    timeout expired in wait_until_healthy

[7459616]  rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_18.04} 2-node-mgr orchestrator_cli}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459616/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459616/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e80ef05513294341aaf1b4ea27b67bfc

    Failed to reconnect to smithi131

[7459629]  rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic}
-----------------------------------------------------------------
time:   00:21:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459629/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459629/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b5a48c55e3b4a37987f1993e3854a90

    timeout expired in wait_until_healthy

[7459637]  rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
-----------------------------------------------------------------
time:   00:43:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459637/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459637/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c1e8bf9814c34ce3b38d23703a11651a

    reached maximum tries (301) after waiting for 300 seconds

[7459640]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm}
-----------------------------------------------------------------
time:   00:18:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459640/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459640/
sentry: https://sentry.ceph.com/organizations/ceph/?query=83f06b5cdd1440deb9a8cad7d0d13a95

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi184 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[7459649]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:23:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459649/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459649/
sentry: https://sentry.ceph.com/organizations/ceph/?query=945e980e3e62422d8a637408bc728e67

    Command failed on smithi195 with status 5: 'sudo systemctl stop
    ceph-5014dc1c-841f-11ee-8dbe-212e2dc638e7@mon.smithi195'

[7459661]  rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:14:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459661/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459661/
sentry: https://sentry.ceph.com/organizations/ceph/?query=458203470827493e8c345fb097c64c40

    Command failed on smithi042 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:ed3c8e926b9de93026c8a7c1d86be5d6821b52ac shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    094a7e80-8420-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json'

[7459667]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459667/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459667/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7d2be6f3d35b413085bea6402016f906

    Failed to reconnect to smithi119

[7459672]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:23:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459672/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459672/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5c236e565c7e4363a5dcf6103515f98d

    Command failed on smithi118 with status 5: 'sudo systemctl stop
    ceph-e8bbe234-8420-11ee-8dbe-212e2dc638e7@mon.smithi118'

[7459675]  rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity}
-----------------------------------------------------------------
time:   00:39:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459675/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459675/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9c7c8f805eeb4f7597ea8604ca4ad453

    Command failed on smithi017 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    27bf0caa-8420-11ee-8dbe-212e2dc638e7 -e
    sha1=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac -- bash -c \'ceph orch
    upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mon
    --hosts $(ceph orch ps | grep mgr.x | awk \'"\'"\'{print $2}\'"\'"\')\''

[7459680]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:26:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459680/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459680/
sentry: https://sentry.ceph.com/organizations/ceph/?query=198cb7f946d84bb6b43ec1405f0dfca5

    Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)

[7459686]  rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:19:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459686/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459686/
sentry: https://sentry.ceph.com/organizations/ceph/?query=948dd88cf051431f878a7b2b98ad15e0

    Command failed on smithi158 with status 5: 'sudo systemctl stop
    ceph-0dfc133e-8421-11ee-8dbe-212e2dc638e7@mon.smithi158'

[7459694]  rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits}
-----------------------------------------------------------------
time:   00:12:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459694/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459694/
sentry: https://sentry.ceph.com/organizations/ceph/?query=89688c83dd364ce2aa0daaf0410e9434

    Command failed (workunit test mon/crush_ops.sh) on smithi136 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh'

[7459703]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:27:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459703/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459703/
sentry: https://sentry.ceph.com/organizations/ceph/?query=00b8d181c81e40e3a853c8362d113e49

    Command failed on smithi059 with status 5: 'sudo systemctl stop
    ceph-99517c56-8423-11ee-8dbe-212e2dc638e7@mon.smithi059'

[7459712]  rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459712/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459712/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a74da247f8e94983a1570f8639ba2905

    Failed to reconnect to smithi119

[7459714]  rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
-----------------------------------------------------------------
time:   00:38:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459714/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459714/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eb5ce33c6b484946a4ad733acc3d9203

    reached maximum tries (301) after waiting for 300 seconds

[7459716]  rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e}
-----------------------------------------------------------------
time:   00:08:23
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459716/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459716/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5190f1f9a8114d75913ae9b173cc51e0

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1
    =ed3c8e926b9de93026c8a7c1d86be5d6821b52ac

[7459719]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:22:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459719/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459719/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7f702d1d10ea4f75a813e187843cfb18

    Command failed on smithi188 with status 5: 'sudo systemctl stop
    ceph-e15d3fe4-8423-11ee-8dbe-212e2dc638e7@mon.smithi188'

[7459726]  rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:13:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459726/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459726/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4fdba3017706466ba538acae2fc9346c

    Command failed on smithi204 with status 127: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:ed3c8e926b9de93026c8a7c1d86be5d6821b52ac shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    4454796e-8424-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json'

[7459734]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep}
-----------------------------------------------------------------
time:   00:20:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459734/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459734/

    "2023-11-16T02:12:32.852059+0000 mon.a (mon.0) 556 : cluster [WRN] Health
    check failed: 1 pool(s) do not have an application enabled
    (POOL_APP_NOT_ENABLED)" in cluster log

[7459737]  rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459737/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459737/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2b915374d61a4da9b12b114286655b51

    Failed to reconnect to smithi069

[7459739]  rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
-----------------------------------------------------------------
time:   00:43:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459739/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459739/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e7c5e75fc27c4c7dbe67534ec270bd9b

    reached maximum tries (301) after waiting for 300 seconds

[7459744]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:19:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459744/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459744/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b5fefa7e3d744913a4896972dc74e57a

    Command failed (workunit test cephadm/test_cephadm.sh) on smithi072 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ed3c8e926b9de93026c8a7c1d86be5d6821b52ac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

[7459748]  rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
-----------------------------------------------------------------
time:   00:48:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459748/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459748/
sentry: https://sentry.ceph.com/organizations/ceph/?query=03c3cddb0d984daea8d3cf3832b02820

    reached maximum tries (301) after waiting for 300 seconds

[7459749]  rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:23:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459749/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459749/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a6f3145a22ca4856b67e71812e889ede

    Command failed on smithi083 with status 5: 'sudo systemctl stop
    ceph-a8ea1bc6-8425-11ee-8dbe-212e2dc638e7@mon.smithi083'

[7459756]  rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:29:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459756/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459756/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f04f1f6c8878475bbd8c666e227e0abc

    timeout expired in wait_until_healthy

[7459760]  rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:36:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459760/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459760/
sentry: https://sentry.ceph.com/organizations/ceph/?query=77bf1006deb34bcf8d4cee6502e05370

    reached maximum tries (301) after waiting for 300 seconds



Pass
=================================================================
[7459336] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:46:23
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459336/

[7459337] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:27:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459337/

[7459338] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench}
time:   01:37:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459338/

[7459339] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start}
time:   00:25:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459339/

[7459341] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery}
time:   00:23:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459341/

[7459342] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw}
time:   00:23:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459342/

[7459343] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:24:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459343/

[7459344] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
time:   00:30:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459344/

[7459345] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:25:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459345/

[7459346] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench}
time:   01:05:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459346/

[7459347] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:37:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459347/

[7459348] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:40:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459348/

[7459349] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:21:30
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459349/

[7459350] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_osdmap_prune}
time:   00:50:50
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459350/

[7459351] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/rados_api_tests}
time:   00:37:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459351/

[7459352] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind}
time:   00:48:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459352/

[7459353] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:40:55
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459353/

[7459354] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:26:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459354/

[7459355] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   00:23:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459355/

[7459356] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls}
time:   00:44:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459356/

[7459357] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/progress}
time:   00:34:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459357/

[7459358] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:29:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459358/

[7459360] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:42:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459360/

[7459361] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect}
time:   00:30:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459361/

[7459362] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start}
time:   00:29:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459362/

[7459363] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:24:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459363/

[7459364] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:54:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459364/

[7459365] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}}
time:   00:33:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459365/

[7459366] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:46:05
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459366/

[7459367] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:40:25
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459367/

[7459368] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:34:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459368/

[7459369] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli}
time:   00:24:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459369/

[7459370] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests}
time:   00:24:23
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459370/

[7459371] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:25:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459371/

[7459372] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write}
time:   00:18:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459372/

[7459373] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:38:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459373/

[7459374] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:27:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459374/

[7459375] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}}
time:   00:24:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459375/

[7459376] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:23:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459376/

[7459377] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:51:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459377/

[7459378] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}}
time:   00:39:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459378/

[7459379] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start}
time:   00:24:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459379/

[7459380] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:14:56
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459380/

[7459381] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object}
time:   00:20:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459381/

[7459383] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon}
time:   00:27:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459383/

[7459384] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery}
time:   00:33:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459384/

[7459385] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}}
time:   00:17:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459385/

[7459386] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:21:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459386/

[7459388] rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_8}}
time:   00:22:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459388/

[7459389] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}}
time:   00:22:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459389/

[7459390] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:22:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459390/

[7459391] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush}
time:   00:21:40
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459391/

[7459392] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04}
time:   02:44:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459392/

[7459393] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest}
time:   00:28:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459393/

[7459395] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459395/

[7459396] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_mon_workunits}
time:   00:45:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459396/

[7459397] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep}
time:   00:33:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459397/

[7459398] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:32:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459398/

[7459399] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read}
time:   00:16:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459399/

[7459400] rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr orchestrator_cli}
time:   00:13:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459400/

[7459401] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/prometheus}
time:   00:15:15
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459401/

[7459402] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:24:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459402/

[7459403] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
time:   00:29:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459403/

[7459404] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi}
time:   00:30:55
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459404/

[7459405] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459405/

[7459406] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:19:50
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459406/

[7459407] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start}
time:   00:16:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459407/

[7459408] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
time:   00:13:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459408/

[7459409] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:16:55
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459409/

[7459410] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final}
time:   00:16:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459410/

[7459411] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-balanced}
time:   00:30:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459411/

[7459412] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:19:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459412/

[7459413] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
time:   00:38:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459413/

[7459414] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1}
time:   00:24:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459414/

[7459416] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:29:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459416/

[7459417] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:19:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459417/

[7459418] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests}
time:   00:42:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459418/

[7459419] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_python}
time:   00:18:55
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459419/

[7459420] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption}
time:   00:13:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459420/

[7459421] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459421/

[7459423] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-localized}
time:   00:29:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459423/

[7459424] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
time:   00:21:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459424/

[7459425] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:22:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459425/

[7459426] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm}
time:   00:18:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459426/

[7459427] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:09:50
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459427/

[7459428] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:18:13
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459428/

[7459429] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews}
time:   00:09:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459429/

[7459430] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start}
time:   00:21:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459430/

[7459431] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}}
time:   00:31:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459431/

[7459432] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects}
time:   00:28:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459432/

[7459434] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read}
time:   00:12:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459434/

[7459435] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:40
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459435/

[7459436] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits}
time:   00:32:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459436/

[7459437] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}}
time:   00:13:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459437/

[7459438] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind}
time:   01:00:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459438/

[7459439] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:17:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459439/

[7459440] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos}
time:   00:09:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459440/

[7459441] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/workunits}
time:   00:14:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459441/

[7459442] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:20:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459442/

[7459443] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:16:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459443/

[7459444] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
time:   00:45:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459444/

[7459445] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}}
time:   00:17:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459445/

[7459446] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:29:21
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459446/

[7459447] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
time:   00:29:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459447/

[7459448] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:30:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459448/

[7459449] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start}
time:   00:20:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459449/

[7459450] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:17:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459450/

[7459451] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch}
time:   00:19:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459451/

[7459452] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code}
time:   00:49:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459452/

[7459453] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:16:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459453/

[7459454] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:12:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459454/

[7459455] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459455/

[7459456] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1}
time:   00:34:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459456/

[7459457] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench}
time:   01:26:56
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459457/

[7459458] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized}
time:   00:31:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459458/

[7459459] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:18:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459459/

[7459460] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:38:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459460/

[7459461] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:52:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459461/

[7459462] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read}
time:   00:12:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459462/

[7459463] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:25:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459463/

[7459464] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:24:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459464/

[7459465] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}}
time:   00:09:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459465/

[7459466] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:16:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459466/

[7459468] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects}
time:   00:32:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459468/

[7459470] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews}
time:   00:08:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459470/

[7459472] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:21:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459472/

[7459473] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:28:15
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459473/

[7459475] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:17:11
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459475/

[7459476] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}}
time:   00:17:50
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459476/

[7459477] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
time:   00:36:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459477/

[7459479] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects}
time:   00:44:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459479/

[7459480] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed}
time:   00:22:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459480/

[7459481] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/crash}
time:   00:19:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459481/

[7459482] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:21:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459482/

[7459483] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:42:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459483/

[7459484] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper}
time:   00:10:17
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459484/

[7459485] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}}
time:   00:46:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459485/

[7459486] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli}
time:   00:25:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459486/

[7459487] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}}
time:   00:16:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459487/

[7459488] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}}
time:   01:13:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459488/

[7459489] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:38:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459489/

[7459490] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:19:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459490/

[7459491] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:17:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459491/

[7459492] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:23:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459492/

[7459493] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start}
time:   00:24:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459493/

[7459494] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:17:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459494/

[7459495] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read}
time:   00:13:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459495/

[7459496] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:19:17
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459496/

[7459497] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon}
time:   00:45:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459497/

[7459498] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-radosbench}
time:   01:31:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459498/

[7459499] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
time:   00:47:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459499/

[7459500] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:51:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459500/

[7459501] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:17:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459501/

[7459502] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459502/

[7459504] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big}
time:   01:24:48
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459504/

[7459505] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:52:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459505/

[7459506] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:51:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459506/

[7459507] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:21:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459507/

[7459508] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start}
time:   00:23:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459508/

[7459509] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final}
time:   00:17:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459509/

[7459510] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr}
time:   00:20:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459510/

[7459511] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
time:   01:01:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459511/

[7459512] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_recovery}
time:   00:43:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459512/

[7459513] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:37:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459513/

[7459514] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big}
time:   00:58:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459514/

[7459515] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic}
time:   00:32:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459515/

[7459516] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small}
time:   00:42:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459516/

[7459517] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:49:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459517/

[7459518] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}}
time:   00:23:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459518/

[7459519] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption}
time:   00:35:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459519/

[7459520] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
time:   00:52:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459520/

[7459521] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write}
time:   00:11:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459521/

[7459522] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete}
time:   00:22:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459522/

[7459523] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/failover}
time:   00:13:30
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459523/

[7459524] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:27:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459524/

[7459525] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:18:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459525/

[7459526] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api}
time:   00:34:34
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459526/

[7459527] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}}
time:   00:22:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459527/

[7459528] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:33:17
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459528/

[7459530] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}}
time:   00:17:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459530/

[7459531] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm}
time:   00:26:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459531/

[7459532] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:41:05
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459532/

[7459533] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}}
time:   02:24:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459533/

[7459534] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}
time:   00:42:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459534/

[7459535] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start}
time:   00:21:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459535/

[7459536] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:42:58
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459536/

[7459537] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:19:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459537/

[7459538] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459538/

[7459539] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced}
time:   00:25:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459539/

[7459540] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps}
time:   00:21:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459540/

[7459541] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos}
time:   00:10:34
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459541/

[7459542] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:30:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459542/

[7459543] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
time:   00:12:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459543/

[7459546] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:16:13
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459546/

[7459547] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:39:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459547/

[7459548] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix}
time:   00:24:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459548/

[7459549] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
time:   00:37:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459549/

[7459550] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write}
time:   00:21:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459550/

[7459551] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}}
time:   00:27:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459551/

[7459552] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced}
time:   00:23:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459552/

[7459553] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:43:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459553/

[7459554] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:22:56
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459554/

[7459555] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start}
time:   00:20:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459555/

[7459556] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews}
time:   00:18:17
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459556/

[7459557] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:19:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459557/

[7459558] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:19:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459558/

[7459559] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:20:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459559/

[7459560] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   00:15:21
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459560/

[7459561] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep}
time:   00:28:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459561/

[7459562] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps}
time:   00:29:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459562/

[7459563] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/insights}
time:   00:12:25
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459563/

[7459564] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:20:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459564/

[7459566] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925}
time:   00:17:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459566/

[7459567] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:14:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459567/

[7459568] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity}
time:   00:47:48
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459568/

[7459569] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench}
time:   01:52:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459569/

[7459571] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc}
time:   00:32:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459571/

[7459572] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:12:13
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459572/

[7459573] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:30
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459573/

[7459574] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start}
time:   00:22:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459574/

[7459575] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache}
time:   00:17:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459575/

[7459576] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:20:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459576/

[7459577] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}}
time:   00:36:58
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459577/

[7459578] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root}
time:   00:48:21
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459578/

[7459579] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}}
time:   02:28:21
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459579/

[7459580] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread}
time:   00:30:34
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459580/

[7459581] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio}
time:   00:13:08
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459581/

[7459582] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:31:11
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459582/

[7459583] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read}
time:   00:26:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459583/

[7459584] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459584/

[7459585] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli}
time:   00:17:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459585/

[7459586] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
time:   00:40:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459586/

[7459588] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects}
time:   00:34:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459588/

[7459590] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:29:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459590/

[7459591] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:56
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459591/

[7459592] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:14:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459592/

[7459593] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:20:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459593/

[7459594] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls}
time:   00:36:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459594/

[7459595] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
time:   00:20:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459595/

[7459596] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start}
time:   00:24:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459596/

[7459597] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews}
time:   00:09:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459597/

[7459598] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon}
time:   00:31:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459598/

[7459599] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests}
time:   00:32:14
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459599/

[7459600] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:18:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459600/

[7459601] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard}
time:   01:07:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459601/

[7459602] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:23:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459602/

[7459603] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04}
time:   03:13:15
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459603/

[7459604] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects}
time:   00:53:40
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459604/

[7459605] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}}
time:   01:02:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459605/

[7459606] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind}
time:   02:14:58
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459606/

[7459607] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/module_selftest}
time:   00:26:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459607/

[7459608] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:18:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459608/

[7459610] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests}
time:   00:35:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459610/

[7459612] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench}
time:   00:13:03
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459612/

[7459614] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency}
time:   00:39:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459614/

[7459615] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/readwrite}
time:   00:17:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459615/

[7459617] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:31:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459617/

[7459618] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:14:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459618/

[7459619] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi}
time:   00:23:56
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459619/

[7459620] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}}
time:   00:57:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459620/

[7459621] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start}
time:   00:15:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459621/

[7459622] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:13:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459622/

[7459623] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final}
time:   00:12:35
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459623/

[7459624] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/radosbench}
time:   00:46:25
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459624/

[7459625] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:30:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459625/

[7459626] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
time:   00:34:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459626/

[7459627] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}}
time:   00:14:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459627/

[7459628] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}}
time:   00:13:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459628/

[7459630] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:15:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459630/

[7459631] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests}
time:   00:41:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459631/

[7459632] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459632/

[7459633] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption}
time:   00:13:15
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459633/

[7459634] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon}
time:   00:50:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459634/

[7459635] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect}
time:   00:14:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459635/

[7459636] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:18:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459636/

[7459638] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}}
time:   01:12:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459638/

[7459639] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read}
time:   00:15:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459639/

[7459641] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:15:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459641/

[7459642] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_recovery}
time:   00:19:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459642/

[7459643] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start}
time:   00:20:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459643/

[7459644] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api}
time:   00:34:43
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459644/

[7459645] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test}
time:   00:17:44
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459645/

[7459646] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests}
time:   00:23:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459646/

[7459647] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/per_module_finisher_stats}
time:   00:17:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459647/

[7459648] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:17:51
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459648/

[7459650] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}
time:   00:15:49
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459650/

[7459651] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
time:   00:49:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459651/

[7459652] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune}
time:   00:56:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459652/

[7459653] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:18:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459653/

[7459654] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos}
time:   00:10:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459654/

[7459655] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:25:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459655/

[7459656] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:13:25
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459656/

[7459657] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
time:   00:38:48
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459657/

[7459658] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python}
time:   00:31:38
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459658/

[7459659] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:15:55
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459659/

[7459660] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object}
time:   00:16:50
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459660/

[7459662] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459662/

[7459663] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start}
time:   00:19:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459663/

[7459664] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:16:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459664/

[7459665] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:09:47
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459665/

[7459666] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects}
time:   00:22:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459666/

[7459668] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw}
time:   00:13:42
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459668/

[7459669] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:16:48
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459669/

[7459670] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}}
time:   00:17:13
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459670/

[7459671] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/set-chunks-read}
time:   00:15:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459671/

[7459673] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}}
time:   00:16:48
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459673/

[7459674] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest}
time:   00:26:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459674/

[7459676] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459676/

[7459677] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:19:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459677/

[7459678] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test}
time:   00:13:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459678/

[7459679] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459679/

[7459681] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start}
time:   00:23:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459681/

[7459682] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}}
time:   00:18:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459682/

[7459683] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced}
time:   00:25:01
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459683/

[7459684] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews}
time:   00:09:21
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459684/

[7459685] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench}
time:   01:04:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459685/

[7459687] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/progress}
time:   00:24:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459687/

[7459688] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:20:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459688/

[7459689] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:32:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459689/

[7459690] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:15:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459690/

[7459691] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}
time:   00:57:33
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459691/

[7459692] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:16:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459692/

[7459693] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
time:   00:37:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459693/

[7459695] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
time:   00:14:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459695/

[7459696] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:36:37
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459696/

[7459697] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd}
time:   03:19:13
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459697/

[7459698] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized}
time:   00:25:09
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459698/

[7459699] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli}
time:   00:18:19
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459699/

[7459700] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459700/

[7459701] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read}
time:   00:12:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459701/

[7459702] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:31:54
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459702/

[7459704] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:07
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459704/

[7459705] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}}
time:   00:17:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459705/

[7459706] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start}
time:   00:23:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459706/

[7459707] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:18:45
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459707/

[7459708] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects}
time:   00:22:16
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459708/

[7459709] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon}
time:   00:27:46
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459709/

[7459710] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:26:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459710/

[7459711] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite}
time:   00:17:28
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459711/

[7459713] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites}
time:   00:31:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459713/

[7459715] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}}
time:   00:18:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459715/

[7459717] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:14:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459717/

[7459718] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:33:02
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459718/

[7459720] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:32:00
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459720/

[7459721] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start}
time:   00:16:06
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459721/

[7459722] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}}
time:   00:30:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459722/

[7459723] rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/rgw 3-final}
time:   00:16:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459723/

[7459724] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}}
time:   00:14:27
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459724/

[7459725] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews}
time:   00:13:34
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459725/

[7459727] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic}
time:   00:58:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459727/

[7459728] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw}
time:   00:12:22
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459728/

[7459729] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:13:20
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459729/

[7459730] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/prometheus}
time:   00:15:41
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459730/

[7459731] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:16:34
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459731/

[7459732] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized}
time:   00:31:24
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459732/

[7459733] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:32:31
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459733/

[7459735] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption}
time:   00:13:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459735/

[7459736] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects}
time:   00:32:52
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459736/

[7459738] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
time:   01:01:26
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459738/

[7459740] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   00:54:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459740/

[7459741] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:18:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459741/

[7459742] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects}
time:   00:29:30
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459742/

[7459743] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:11:57
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459743/

[7459745] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests}
time:   00:27:39
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459745/

[7459746] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start}
time:   00:19:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459746/

[7459747] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}}
time:   00:34:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459747/

[7459750] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1}
time:   00:34:05
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459750/

[7459751] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos}
time:   00:09:32
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459751/

[7459752] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}}
time:   00:26:36
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459752/

[7459753] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:18:12
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459753/

[7459754] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api}
time:   00:34:59
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459754/

[7459755] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub}
time:   01:46:18
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459755/

[7459757] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   00:20:05
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459757/

[7459758] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write}
time:   00:12:53
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459758/

[7459759] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:29:04
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459759/

[7459761] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:33:10
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459761/

[7459762] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:43:29
info:   https://pulpito.ceph.com/yuriw-2023-11-15_21:10:46-rados-wip-yuri3-testing-2023-11-14-1227-pacific-distro-default-smithi/7459762/