2023-04-06T00:13:59.060 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-06T00:13:59.061 INFO:teuthology.results:subset: '111/120000' 2023-04-06T00:13:59.061 INFO:teuthology.results:seed: '5090' 2023-04-06T10:41:12.609 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-06T10:41:12.644 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-04-06T21:49:23.067 INFO:teuthology.results:Tests finished! gathering results... 2023-04-06T21:49:55.945 INFO:scrape:Found 430 jobs 2023-04-06T21:49:57.559 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233151/teuthology.log 2023-04-06T21:49:59.998 INFO:scrape:Found 12 distinct failure reasons 2023-04-06T21:49:59.999 INFO:scrape:Failure: No module named 'tasks.ceph' 2023-04-06T21:49:59.999 INFO:scrape:2 jobs: ['7233510', '7233519'] 2023-04-06T21:50:00.000 INFO:scrape:suites intersection: [] 2023-04-06T21:50:00.000 INFO:scrape:suites union: ['1-start', '2-thrash', '3-tasks/small-objects', 'fixed-2', 'mon_election/connectivity', 'msgr/async-v2only', 'rados', 'rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'rados/singleton-nomsgr/{all/multi-backfill-reject', 'root}', 'supported-random-distro$/{centos_8}}'] 2023-04-06T21:50:00.000 INFO:scrape: 2023-04-06T21:50:00.000 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-04-06T21:50:00.001 INFO:scrape:1 jobs: ['7233150'] 2023-04-06T21:50:00.001 INFO:scrape:suites: ['rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}'] 2023-04-06T21:50:00.001 INFO:scrape: 2023-04-06T21:50:00.002 INFO:scrape:Dead: b'2023-04-06T16:00:10.094 DEBUG:teuthology.exit:Finished running handlers' 2023-04-06T21:50:00.002 INFO:scrape:7 jobs: ['7233263', '7233102', '7233430', '7233370', '7233302', '7233477', '7233444'] 2023-04-06T21:50:00.002 INFO:scrape:suites intersection: ['3-upgrade-with-workload'] 2023-04-06T21:50:00.003 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-bootstrap/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '3-inline/no', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'roles', 'tasks/{0-from/v16.2.4'] 2023-04-06T21:50:00.003 INFO:scrape: 2023-04-06T21:50:00.003 INFO:scrape:Failure: Cannot connect to remote host smithi097 2023-04-06T21:50:00.004 INFO:scrape:8 jobs: ['7233495', '7233283', '7233148', '7233470', '7233117', '7233443', '7233425', '7233178'] 2023-04-06T21:50:00.004 INFO:scrape:suites intersection: [] 2023-04-06T21:50:00.004 INFO:scrape:suites union: ['0-nvme-loop', '1-install/luminous', '1-install/mimic', '1-install/mimic-v1only', '1-install/nautilus', '1-start', '2-ops/rm-zap-flag}', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-pacific', '8-workload/{rbd-python', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'bluestore-bitmap', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'distro/ubuntu_18.04', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'ubuntu_18.04}', 'workloads/cache-snaps}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/test_rbd_api}'] 2023-04-06T21:50:00.005 INFO:scrape: 2023-04-06T21:50:00.005 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi139.front.sepia.ceph.com: ['type=AVC msg=audit(1680774407.344:19188): avc: denied { ioctl } for pid=157146 comm="iptables" path="/var/lib/containers/storage/overlay/a25cba64c824ab38ae87a30ecb9f09e4cb6b88eb1c2ecd81b9f0d3e3ddfc119d/merged" dev="overlay" ino=3412123 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2023-04-06T21:50:00.005 INFO:scrape:2 jobs: ['7233398', '7233502'] 2023-04-06T21:50:00.005 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2023-04-06T21:50:00.006 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] 2023-04-06T21:50:00.006 INFO:scrape: 2023-04-06T21:50:00.006 INFO:scrape:Failure: Command failed on smithi139 with status 1: 'sudo yum install -y kernel' 2023-04-06T21:50:00.007 INFO:scrape:15 jobs: ['7233514', '7233365', '7233254', '7233139', '7233306', '7233355', '7233319', '7233451', '7233231', '7233118', '7233388', '7233507', '7233405', '7233253', '7233520'] 2023-04-06T21:50:00.007 INFO:scrape:suites intersection: [] 2023-04-06T21:50:00.007 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-bootstrap/octopus', '1-pg-log-overrides/short_pg_log', '1-ranks/1', '1-rook', '1-start', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-recovery-overrides/{more-active-recovery}', '2-services/rgw', '2-thrash', '2-workload/none', '2-workload/radosbench', '3-final', '3-final}', '3-inline/yes', '3-scrub-overrides/{default}', '3-tasks/rados_api_tests', '3-upgrade-with-workload', '4-final}', '4-verify}', '4-verify}}', 'backoff/peering', 'centos_8.stream_container_tools', 'ceph', 'cluster/1-node', 'cluster/3-node', 'clusters/{fixed-2', 'clusters/{fixed-4', 'conf/{client', 'crc-failures/default', 'd-balancer/on', 'fixed-2', 'ignorelist_wrongly_marked_down', 'k8s/1.21', 'mds', 'mon', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async', 'msgr/async-v2only', 'net/calico', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados', 'rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/thrash-erasure-code-shec/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{default}', 'roles', 'rook/1.6.2}', 'rook/master}', 'root}', 'supported-random-distro$/{centos_8}', 'task/test_e2e}', 'task/test_orch_cli_mon}', 'tasks/{0-from/v16.2.4', 'thrashers/default', 'thrashosds-health', 'workloads/cache-snaps}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}'] 2023-04-06T21:50:00.008 INFO:scrape: 2023-04-06T21:50:00.008 INFO:scrape:Failure: Command failed (workunit test cephadm/create_iscsi_disks.sh) on smithi181 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/cephadm/create_iscsi_disks.sh' 2023-04-06T21:50:00.008 INFO:scrape:2 jobs: ['7233474', '7233260'] 2023-04-06T21:50:00.008 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2023-04-06T21:50:00.009 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2023-04-06T21:50:00.009 INFO:scrape: 2023-04-06T21:50:00.009 INFO:scrape:Failure: Command failed on smithi173 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:681dd7280d36cda3abcedb40f86f672671cd2e37 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66f93650-d446-11ed-9aff-001a4aab830c -- ceph mon dump -f json' 2023-04-06T21:50:00.010 INFO:scrape:1 jobs: ['7233316'] 2023-04-06T21:50:00.010 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-services/nfs-ingress-rgw-user', '3-final}', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0'] 2023-04-06T21:50:00.010 INFO:scrape: 2023-04-06T21:50:00.010 INFO:scrape:Dead: None 2023-04-06T21:50:00.011 INFO:scrape:1 jobs: ['7233151'] 2023-04-06T21:50:00.011 INFO:scrape: 2023-04-06T21:50:00.011 INFO:scrape:Failure: Command failed on smithi142 with status 1: 'test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm' 2023-04-06T21:50:00.012 INFO:scrape:1 jobs: ['7233129'] 2023-04-06T21:50:00.012 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4'] 2023-04-06T21:50:00.012 INFO:scrape: 2023-04-06T21:50:00.012 INFO:scrape:Failure: Command failed on smithi006 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:681dd7280d36cda3abcedb40f86f672671cd2e37 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dba801aa-d439-11ed-9aff-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2023-04-06T21:50:00.013 INFO:scrape:1 jobs: ['7233286'] 2023-04-06T21:50:00.013 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2023-04-06T21:50:00.013 INFO:scrape: 2023-04-06T21:50:00.014 INFO:scrape:Failure: Command failed (workunit test mon/crush_ops.sh) on smithi067 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh' 2023-04-06T21:50:00.014 INFO:scrape:1 jobs: ['7233452'] 2023-04-06T21:50:00.014 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/classic', 'msgr-failures/mon-delay', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/one', 'workloads/rados_mon_workunits}'] 2023-04-06T21:50:00.014 INFO:scrape: 2023-04-06T21:50:00.034 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/ failed: 34 dead: 8 running: 0 waiting: 0 queued: 0 passed: 388 Fail ================================================================= [7233117] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233117/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233117/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ebc98a681d1421d979fe4b57206a7e9 Cannot connect to remote host smithi149 [7233118] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233118/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233118/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cd7a7434f0fd44a79d946f327e9f70a2 Command failed on smithi148 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7233129] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 01:14:25 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233129/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233129/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58c295961ce8480e9287515d57f0142b Command failed on smithi142 with status 1: 'test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm' [7233139] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233139/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233139/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2fa8a3b5ae834114a89f3680f7c3be0d Command failed on smithi181 with status 1: 'sudo yum install -y kernel' [7233148] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233148/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233148/ sentry: https://sentry.ceph.com/organizations/ceph/?query=23470fa503ff484884d8db620cbd7e44 Cannot connect to remote host smithi177 [7233150] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} ----------------------------------------------------------------- time: 01:45:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233150/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233150/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f62dc9eb683942e48dd83f232ad049ce Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7233178] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233178/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233178/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c7c15074f0be4094ab94b140755841b4 Cannot connect to remote host smithi084 [7233231] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233231/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233231/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6c5764bbcdf94a3b8be32209d58a06e1 Command failed on smithi187 with status 1: 'sudo yum install -y kernel' [7233253] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233253/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233253/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9621216edbee4f2ba23a6f144f82708d Command failed on smithi049 with status 1: 'sudo yum install -y kernel' [7233254] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:05:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233254/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233254/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ccfb52ccc7be452fad38b9eb2d882028 Command failed on smithi149 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7233260] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:15:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233260/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233260/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cf9b53ea99a24ccbb9c1c9f801b85ce0 Command failed (workunit test cephadm/create_iscsi_disks.sh) on smithi120 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.1/qa/workunits/cephadm/create_iscsi_disks.sh' [7233283] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233283/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233283/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d181b1c4a1274cd8b62cceb492f851b5 Cannot connect to remote host smithi088 [7233286] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233286/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233286/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ede42eda739a4732bcdc8a6c7342b11d Command failed on smithi006 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:681dd7280d36cda3abcedb40f86f672671cd2e37 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dba801aa-d439-11ed-9aff-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' [7233306] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233306/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233306/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9f965898b694af1860330108729e44d Command failed on smithi186 with status 1: 'sudo yum install -y kernel' [7233316] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:11:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233316/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233316/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f361444177ac4f1e87899228f7bd3d7f Command failed on smithi173 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:681dd7280d36cda3abcedb40f86f672671cd2e37 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66f93650-d446-11ed-9aff-001a4aab830c -- ceph mon dump -f json' [7233319] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233319/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233319/ sentry: https://sentry.ceph.com/organizations/ceph/?query=09eb89000c2f48b9bee638ae9d57f8f9 Command failed on smithi082 with status 1: 'sudo yum install -y kernel' [7233355] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233355/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233355/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02053d0717b74421a3f2c472d0bf3b48 Command failed on smithi006 with status 1: 'sudo yum install -y kernel' [7233365] rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233365/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233365/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5fb50723fd0249ce9cb1680021b926ca Command failed on smithi088 with status 1: 'sudo yum install -y kernel' [7233388] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:06:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233388/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233388/ sentry: https://sentry.ceph.com/organizations/ceph/?query=561debb4bc414196a5f8aa1621427137 Command failed on smithi049 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7233398] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 01:15:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233398/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233398/ SELinux denials found on ubuntu@smithi139.front.sepia.ceph.com: ['type=AVC msg=audit(1680774407.344:19188): avc: denied { ioctl } for pid=157146 comm="iptables" path="/var/lib/containers/storage/overlay/a25cba64c824ab38a e87a30ecb9f09e4cb6b88eb1c2ecd81b9f0d3e3ddfc119d/merged" dev="overlay" ino=3412123 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7233405] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233405/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233405/ sentry: https://sentry.ceph.com/organizations/ceph/?query=952a1f273a4146ae83019df7dd28cb8d Command failed on smithi178 with status 1: 'sudo yum install -y kernel' [7233425] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233425/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233425/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0a8943bc2fc8488e9fa7f66b25ce2484 Cannot connect to remote host smithi097 [7233443] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233443/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233443/ sentry: https://sentry.ceph.com/organizations/ceph/?query=611a48aa1f9b4931afc4198f6fad446b Cannot connect to remote host smithi153 [7233451] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233451/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233451/ sentry: https://sentry.ceph.com/organizations/ceph/?query=05344f0d215945d49418b5138c783785 Command failed on smithi138 with status 1: 'sudo yum install -y kernel' [7233452] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 00:15:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233452/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233452/ sentry: https://sentry.ceph.com/organizations/ceph/?query=81c566416a3a437390cefaed55045ce5 Command failed (workunit test mon/crush_ops.sh) on smithi067 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh' [7233470] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233470/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233470/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5dfde6ec394e4725b6dc9b8e10e0b3f0 Cannot connect to remote host smithi027 [7233474] rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233474/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233474/ sentry: https://sentry.ceph.com/organizations/ceph/?query=91dbdbc61bc84563a6040c5031cb0d7f Command failed (workunit test cephadm/create_iscsi_disks.sh) on smithi181 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=681dd7280d36cda3abcedb40f86f672671cd2e37 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.1/qa/workunits/cephadm/create_iscsi_disks.sh' [7233495] rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233495/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233495/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4f51b4166b32498c8ddcad65300d2f33 Cannot connect to remote host smithi097 [7233502] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 01:16:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233502/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233502/ SELinux denials found on ubuntu@smithi035.front.sepia.ceph.com: ['type=AVC msg=audit(1680782167.220:19182): avc: denied { ioctl } for pid=157336 comm="iptables" path="/var/lib/containers/storage/overlay/9196da7322ae834e3 7b23187efaa343b1794e62d20d88128ab4e17acc8e3f237/merged" dev="overlay" ino=3412123 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7233507] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233507/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233507/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b5ff1366c52f43489e5c4d34253c7e24 Command failed on smithi174 with status 1: 'sudo yum install -y kernel' [7233510] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 01:09:27 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233510/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233510/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c679c41db55549029703b5c440082952 No module named 'tasks.ceph' [7233514] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233514/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233514/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6d8c5c4ce71a4f0194cac88ef181cda5 Command failed on smithi139 with status 1: 'sudo yum install -y kernel' [7233519] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} ----------------------------------------------------------------- time: 01:11:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233519/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233519/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d0aaa5670ef448a2aeda95c6e640b83d No module named 'tasks.cephadm' [7233520] rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} ----------------------------------------------------------------- time: 00:06:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233520/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233520/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cc929723777549e28f291acb07842497 Command failed on smithi043 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' Dead ================================================================= [7233102] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233102/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233102/ hit max job timeout [7233151] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233151/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233151/ Error reimaging machines: 'NoneType' object has no attribute '_fields' [7233263] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233263/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233263/ hit max job timeout [7233302] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233302/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233302/ hit max job timeout [7233370] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233370/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233370/ hit max job timeout [7233430] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233430/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233430/ hit max job timeout [7233444] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233444/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233444/ hit max job timeout [7233477] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233477/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233477/ hit max job timeout Pass ================================================================= [7233092] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233092/ [7233093] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:25:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233093/ [7233094] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:18:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233094/ [7233095] rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} time: 04:48:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233095/ [7233096] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 01:19:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233096/ [7233097] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233097/ [7233098] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_pids_limit} time: 01:16:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233098/ [7233099] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:15:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233099/ [7233100] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233100/ [7233101] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 01:04:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233101/ [7233103] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:30:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233103/ [7233104] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233104/ [7233105] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:41:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233105/ [7233106] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:26:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233106/ [7233107] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:11:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233107/ [7233108] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 01:34:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233108/ [7233109] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233109/ [7233110] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233110/ [7233111] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:11:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233111/ [7233112] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:35:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233112/ [7233113] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233113/ [7233114] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 01:23:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233114/ [7233115] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 02:03:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233115/ [7233116] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 01:16:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233116/ [7233119] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233119/ [7233120] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:32:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233120/ [7233121] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 01:16:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233121/ [7233122] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 01:39:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233122/ [7233123] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:01 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233123/ [7233124] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:33:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233124/ [7233125] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 01:17:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233125/ [7233126] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:13:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233126/ [7233127] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233127/ [7233128] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233128/ [7233130] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:16:17 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233130/ [7233131] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233131/ [7233132] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 01:26:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233132/ [7233133] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:52:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233133/ [7233134] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:31:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233134/ [7233135] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:23:11 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233135/ [7233136] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:27:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233136/ [7233137] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233137/ [7233138] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233138/ [7233140] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 01:28:13 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233140/ [7233141] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:08:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233141/ [7233142] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233142/ [7233143] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} tasks/dashboard} time: 02:06:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233143/ [7233144] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:08:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233144/ [7233145] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233145/ [7233146] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233146/ [7233147] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:16:35 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233147/ [7233149] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 01:23:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233149/ [7233152] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:27:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233152/ [7233153] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/workunits} time: 01:17:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233153/ [7233154] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 01:24:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233154/ [7233155] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:44:25 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233155/ [7233156] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:19:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233156/ [7233157] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr orchestrator_cli} time: 01:18:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233157/ [7233158] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233158/ [7233159] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:22:27 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233159/ [7233160] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 02:21:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233160/ [7233161] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:11 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233161/ [7233162] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233162/ [7233163] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233163/ [7233164] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:08:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233164/ [7233165] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:29:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233165/ [7233166] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:25:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233166/ [7233167] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:22:48 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233167/ [7233168] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 01:24:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233168/ [7233169] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 01:35:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233169/ [7233170] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 01:29:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233170/ [7233171] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:40:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233171/ [7233172] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233172/ [7233173] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233173/ [7233174] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:39:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233174/ [7233175] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:49:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233175/ [7233176] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 01:13:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233176/ [7233177] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 01:24:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233177/ [7233179] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:26:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233179/ [7233180] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:15:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233180/ [7233181] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:18:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233181/ [7233182] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} time: 01:13:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233182/ [7233183] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233183/ [7233184] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233184/ [7233185] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 01:12:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233185/ [7233186] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:18:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233186/ [7233187] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 01:39:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233187/ [7233188] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:24:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233188/ [7233189] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:34:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233189/ [7233190] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233190/ [7233191] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:27:52 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233191/ [7233192] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233192/ [7233193] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits} time: 01:31:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233193/ [7233194] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233194/ [7233195] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 01:57:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233195/ [7233196] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:15:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233196/ [7233197] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:22:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233197/ [7233198] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233198/ [7233199] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:08:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233199/ [7233200] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 01:50:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233200/ [7233201] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233201/ [7233202] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 01:30:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233202/ [7233203] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233203/ [7233204] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233204/ [7233205] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:17:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233205/ [7233206] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:16:25 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233206/ [7233207] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:53:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233207/ [7233208] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:42:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233208/ [7233209] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_pids_limit} time: 01:14:49 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233209/ [7233210] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233210/ [7233211] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:10:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233211/ [7233212] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:30:06 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233212/ [7233213] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:24:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233213/ [7233214] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 01:31:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233214/ [7233215] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233215/ [7233216] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:31:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233216/ [7233217] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:52:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233217/ [7233218] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233218/ [7233219] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233219/ [7233220] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:22:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233220/ [7233221] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233221/ [7233222] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:10:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233222/ [7233223] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 01:28:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233223/ [7233224] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:29:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233224/ [7233225] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233225/ [7233226] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:08:11 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233226/ [7233227] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:34:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233227/ [7233228] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:25:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233228/ [7233229] rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:41:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233229/ [7233230] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233230/ [7233232] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:06 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233232/ [7233233] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 01:34:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233233/ [7233234] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 01:20:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233234/ [7233235] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:13:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233235/ [7233236] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:37:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233236/ [7233237] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233237/ [7233238] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233238/ [7233239] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:41:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233239/ [7233240] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_striper} time: 00:46:27 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233240/ [7233241] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:32:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233241/ [7233242] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 01:14:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233242/ [7233243] rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233243/ [7233244] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 01:12:17 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233244/ [7233245] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:30:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233245/ [7233246] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 01:15:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233246/ [7233247] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233247/ [7233248] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:13:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233248/ [7233249] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:23:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233249/ [7233250] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:28:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233250/ [7233251] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233251/ [7233252] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:16:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233252/ [7233255] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 00:57:01 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233255/ [7233256] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:33:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233256/ [7233257] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:49:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233257/ [7233258] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:16:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233258/ [7233259] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:10:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233259/ [7233261] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 01:18:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233261/ [7233262] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233262/ [7233264] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 01:22:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233264/ [7233265] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:29:28 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233265/ [7233266] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:15:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233266/ [7233267] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} time: 00:15:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233267/ [7233268] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 01:38:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233268/ [7233269] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:18:25 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233269/ [7233270] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233270/ [7233271] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:31:35 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233271/ [7233272] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233272/ [7233273] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233273/ [7233274] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 01:24:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233274/ [7233275] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:29:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233275/ [7233276] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 01:09:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233276/ [7233277] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 01:34:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233277/ [7233278] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233278/ [7233279] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233279/ [7233280] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:18:43 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233280/ [7233281] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:27 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233281/ [7233282] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:16:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233282/ [7233284] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:15:01 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233284/ [7233285] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 01:49:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233285/ [7233287] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:12:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233287/ [7233288] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} time: 01:13:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233288/ [7233289] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233289/ [7233290] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:30:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233290/ [7233291] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 01:23:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233291/ [7233292] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:18:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233292/ [7233293] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:35:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233293/ [7233294] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:17:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233294/ [7233295] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:28:25 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233295/ [7233296] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 01:22:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233296/ [7233297] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 01:23:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233297/ [7233298] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:22:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233298/ [7233299] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:27:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233299/ [7233300] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 01:06:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233300/ [7233301] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:35:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233301/ [7233303] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:28:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233303/ [7233304] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233304/ [7233305] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:23:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233305/ [7233307] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233307/ [7233308] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 01:40:49 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233308/ [7233309] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 01:28:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233309/ [7233310] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:39:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233310/ [7233311] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:23:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233311/ [7233312] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} time: 00:19:01 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233312/ [7233313] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233313/ [7233314] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_pids_limit} time: 01:14:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233314/ [7233315] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233315/ [7233317] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233317/ [7233318] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 01:25:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233318/ [7233320] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:30:35 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233320/ [7233321] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233321/ [7233322] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:29:06 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233322/ [7233323] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233323/ [7233324] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:13:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233324/ [7233325] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:59:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233325/ [7233326] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:49:13 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233326/ [7233327] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} time: 01:28:57 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233327/ [7233328] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} time: 00:34:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233328/ [7233329] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 01:15:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233329/ [7233330] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233330/ [7233331] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:21:06 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233331/ [7233332] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} time: 00:15:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233332/ [7233333] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:19:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233333/ [7233334] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:30:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233334/ [7233335] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 01:52:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233335/ [7233336] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:39:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233336/ [7233337] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:33:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233337/ [7233338] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233338/ [7233339] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233339/ [7233340] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:27 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233340/ [7233341] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233341/ [7233342] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} time: 01:16:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233342/ [7233343] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:32:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233343/ [7233344] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:19:46 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233344/ [7233345] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233345/ [7233346] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:37:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233346/ [7233347] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233347/ [7233348] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233348/ [7233349] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:27:23 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233349/ [7233350] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 01:24:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233350/ [7233351] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:35:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233351/ [7233352] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 01:12:11 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233352/ [7233353] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} time: 00:24:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233353/ [7233354] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:11:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233354/ [7233356] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 01:33:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233356/ [7233357] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233357/ [7233358] rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} tasks/dashboard} time: 02:01:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233358/ [7233359] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:20:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233359/ [7233360] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:53:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233360/ [7233361] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:52:11 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233361/ [7233362] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 01:21:50 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233362/ [7233363] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 03:15:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233363/ [7233364] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 01:18:43 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233364/ [7233366] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/progress} time: 00:18:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233366/ [7233367] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:30:49 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233367/ [7233368] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:28:57 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233368/ [7233369] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:48 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233369/ [7233371] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 01:02:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233371/ [7233372] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233372/ [7233373] rados/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr orchestrator_cli} time: 01:19:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233373/ [7233374] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233374/ [7233375] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:47 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233375/ [7233376] rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 02:46:15 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233376/ [7233377] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 01:01:29 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233377/ [7233378] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} time: 00:29:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233378/ [7233379] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:27:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233379/ [7233380] rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:25:57 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233380/ [7233381] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench} time: 00:58:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233381/ [7233382] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:24:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233382/ [7233383] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 01:31:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233383/ [7233384] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:11:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233384/ [7233385] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 01:09:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233385/ [7233386] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:31:51 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233386/ [7233387] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233387/ [7233389] rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:40:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233389/ [7233390] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233390/ [7233391] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} time: 01:09:17 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233391/ [7233392] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 00:47:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233392/ [7233393] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:17:20 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233393/ [7233394] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233394/ [7233395] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:19:28 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233395/ [7233396] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 02:05:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233396/ [7233397] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233397/ [7233399] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233399/ [7233400] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 01:12:35 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233400/ [7233401] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} time: 00:17:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233401/ [7233402] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:33:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233402/ [7233403] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:16:56 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233403/ [7233404] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:19:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233404/ [7233406] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:30:54 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233406/ [7233407] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 01:13:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233407/ [7233408] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233408/ [7233409] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:55:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233409/ [7233410] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:00:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233410/ [7233411] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233411/ [7233412] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} time: 00:22:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233412/ [7233413] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:10 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233413/ [7233414] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:06:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233414/ [7233415] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 01:41:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233415/ [7233416] rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:30:38 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233416/ [7233417] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:17 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233417/ [7233418] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:16:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233418/ [7233419] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233419/ [7233420] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233420/ [7233421] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233421/ [7233422] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233422/ [7233423] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233423/ [7233424] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:21:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233424/ [7233426] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233426/ [7233427] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_pids_limit} time: 01:17:08 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233427/ [7233428] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233428/ [7233429] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233429/ [7233431] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233431/ [7233432] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 01:23:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233432/ [7233433] rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} time: 00:52:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233433/ [7233434] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 01:26:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233434/ [7233435] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233435/ [7233436] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:13:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233436/ [7233437] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:04 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233437/ [7233438] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} time: 01:27:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233438/ [7233439] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:21:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233439/ [7233440] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 01:13:15 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233440/ [7233441] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233441/ [7233442] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 01:10:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233442/ [7233445] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233445/ [7233446] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:32:09 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233446/ [7233447] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:27:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233447/ [7233448] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 01:51:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233448/ [7233449] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:08:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233449/ [7233450] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/workunits} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233450/ [7233453] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233453/ [7233454] rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 01:37:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233454/ [7233455] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 03:13:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233455/ [7233456] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 01:21:24 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233456/ [7233457] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} time: 01:17:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233457/ [7233458] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233458/ [7233459] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:10:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233459/ [7233460] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233460/ [7233461] rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:30:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233461/ [7233462] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233462/ [7233463] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 01:10:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233463/ [7233464] rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:23:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233464/ [7233465] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:16:42 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233465/ [7233466] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:28:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233466/ [7233467] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} time: 01:26:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233467/ [7233468] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:35:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233468/ [7233469] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:51:45 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233469/ [7233471] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:25:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233471/ [7233472] rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233472/ [7233473] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:14:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233473/ [7233475] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 01:10:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233475/ [7233476] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233476/ [7233478] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:12 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233478/ [7233479] rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} time: 00:29:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233479/ [7233480] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:30:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233480/ [7233481] rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:14:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233481/ [7233482] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:10:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233482/ [7233483] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233483/ [7233484] rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:55:32 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233484/ [7233485] rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} time: 00:58:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233485/ [7233486] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233486/ [7233487] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 01:17:00 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233487/ [7233488] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:05 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233488/ [7233489] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 01:28:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233489/ [7233490] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:27:34 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233490/ [7233491] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 01:18:55 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233491/ [7233492] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} time: 01:09:26 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233492/ [7233493] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/crash} time: 01:11:02 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233493/ [7233494] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/snaps-few-objects} time: 01:24:21 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233494/ [7233496] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:33:18 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233496/ [7233497] rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233497/ [7233498] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:55:13 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233498/ [7233499] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233499/ [7233500] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:28:31 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233500/ [7233501] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 01:12:41 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233501/ [7233503] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:28:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233503/ [7233504] rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} time: 00:17:36 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233504/ [7233505] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:35:59 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233505/ [7233506] rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:20:44 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233506/ [7233508] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 01:28:58 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233508/ [7233509] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} time: 00:22:33 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233509/ [7233511] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:53 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233511/ [7233512] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:34:16 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233512/ [7233513] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:45:22 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233513/ [7233515] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:38:14 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233515/ [7233516] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:10:47 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233516/ [7233517] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233517/ [7233518] rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:29:52 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233518/ [7233521] rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:38:40 info: http://pulpito.ceph.com/yuriw-2023-04-05_20:32:43-rados-wip-yuri8-testing-2023-03-31-0812-pacific-distro-default-smithi/7233521/